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Why  GAO  Did  This  Study 

Most  veterans  transition  to  civilian  life 
trouble-free.  For  those  who  struggle 
with  their  transition  to  the  point  that 
they  are  arrested  and  jailed,  VA 
created  the  VJO  Program,  which 
connects  veterans  with  supports  and 
services  to  help  avoid  re-incarceration. 
The  program  relies  on  VJO  specialists 
to  link  veterans  to  treatment.  GAO  was 
asked  to  review  the  management  of 
the  VJO  Program. 

This  report  examines  1 )  how  the 
program  delivers  services  and  the 
number  and  characteristics  of  veterans 
in  the  program,  2)  the  extent  to  which 
VA  uses  performance  assessment  of 
the  program,  and  3)  the  key  challenges 
VA  has  identified  and  the  extent  to 
which  VA  has  developed  mitigation 
strategies.  GAO  obtained  VA  data  on 
program  participants  for  fiscal  years 
2012  through  2015;  reviewed 
documents;  interviewed  VA  officials 
and  staff  from  nine  areas  served  by  a 
VA  medical  center  and  selected  for 
their  geographic  diversity  and 
differences  in  the  structures  of  local 
criminal  justice  systems;  and  in  three 
of  the  areas  interviewed  criminal 
justice  system  stakeholders  and 
veterans.  While  information  from  these 
interviews  cannot  be  generalized,  they 
provide  insights  on  program  challenges 
and  operations. 

What  GAO  Recommends 

To  improve  program  management,  VA 
should  establish  performance  goals 
and  measures  and  conduct  a 
comprehensive  risk  assessment.  In 
commenting  on  a  draft  of  this  report, 

VA  agreed  with  the  recommendations 
and  discussed  actions  it  plans  to  take 
to  implement  them. 

View  GAO-16-393.  For  more  information, 
contact  Daniel  Bertoni  at  (202)  512-7215  or 
bertonid@gao.gov. 


April  2016 


VETERANS  JUSTICE  OUTREACH  PROGRAM 

VA  Could  Improve  Management  by  Establishing 
Performance  Measures  and  Fully  Assessing  Risks 


What  GAO  Found 


The  Veterans  Justice  Outreach  (VJO)  Program — created  by  the  Department  of 
Veterans  Affairs  (VA) — operates  through  VA  medical  centers  to  provide  services 
to  veterans  involved  in  local  criminal  justice  systems,  and  in  fiscal  year  2015 
served  about  46,500  veterans,  mostly  men  and  many  diagnosed  with  mental 
health  or  substance  abuse  problems.  Officials  from  VA  medical  centers  manage 
more  than  260  VJO  Program  specialists  who  identify  veterans  in  jails  and  local 
courts,  assess  their  health  and  social  needs,  and  link  them  to  supports  and 
services.  VJO  specialists  monitor  veterans’  services  and  treatment  in  courts 
dedicated  to  veteran  offenders.  According  to  VA  data,  the  number  of  veterans 
served  by  the  program  increased  72  percent  from  fiscal  years  2012  -  2015.  In 
addition,  many  veterans  involved  in  the  program  were  Post-9/11  veterans;  about 
two-thirds  were  diagnosed  with  one  or  more  mental  health  problems. 

VA  has  taken  some  steps  to  incorporate  a  performance  assessment  system  into 
the  VJO  Program,  one  component  of  effective  program  management  (see 
figure).  Specifically,  VA  developed  strategic  goals  and  plans  to  conduct 
evaluations.  However,  VA  has  not  established  performance  goals  with  related 
targets,  timeframes,  and  performance  measures  for  any  of  the  program’s  five 
broad  strategic  goals.  VA  officials  told  GAO  they  have  not  taken  this  step,  in  part, 
because  VA  medical  centers  have  flexibility  in  determining  the  activities  of  VJO 
specialists.  GAO’s  past  work  has  highlighted  strategies  that  agencies  can  use  in 
this  situation,  such  as  developing  measures  based  on  common  activities.  Best 
practices  call  for  agencies  to  establish  performance  goals  and  associated 
performance  measures.  Until  VA  incorporates  performance  goals  and  measures, 
it  will  lack  a  systematic  way  to  obtain  ongoing  information  to  identify  possible 
underperforming  areas  for  improvements. 


Key  Elements  of  Program  Performance  Assessment  Systems 
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VA  identified  several  key  challenges — most  of  which  were  related  to  the  demand 
for  services  outpacing  the  program’s  resources — but  has  not  fully  developed 
appropriate  mitigation  strategies.  One  key  challenge,  for  example,  is  addressing 
increased  program  demand  as  jail  administrators  more  widely  use  VA’s  online 
system  that  better  identifies  incarcerated  veterans.  In  addition,  a  major  reason  for 
the  demand-resource  imbalance  is  the  heavier  workload  of  VJO  specialists 
serving  veterans  in  an  expanding  number  of  courts  dedicated  to  veterans, 
according  to  VA  officials  and  stakeholders  that  GAO  interviewed.  However,  GAO 
found  that  VA  did  not  comprehensively  identify  and  assess  risks  posed  by  each 
of  the  key  challenges  it  identified,  contrary  to  federal  internal  control  standards. 
Absent  a  comprehensive  risk  assessment,  VA  is  not  well-positioned  to  develop 
appropriate  strategies  to  mitigate  the  greatest  risks,  which  may  limit  its  ability  to 
help  justice-involved  veterans  receive  assistance  and  avoid  re-incarceration. 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


April  28,  2016 

The  Honorable  Johnny  Isakson 
Chairman 

The  Honorable  Richard  Blumenthal 
Ranking  Member 
Committee  on  Veterans’  Affairs 
United  States  Senate 

The  Honorable  Corrine  Brown 
Ranking  Member 
Committee  on  Veterans’  Affairs 
House  of  Representatives 

While  most  veterans  transition  to  civilian  life  relatively  trouble-free,  others 
face  difficulties,  such  as  mental  illness,  substance  abuse,  unemployment, 
homelessness,  and  other  serious  issues.  For  some  veterans,  these 
issues  may  go  unaddressed  and  may  lead  them  to  commit  crimes  and 
other  offenses.1  To  help  veterans  who  become  involved  with  their  local 
criminal  justice  systems  (referred  to  as  “justice-involved  veterans”),  the 
Department  of  Veterans  Affairs  (VA)  established  the  Veterans  Justice 
Outreach  (VJO)  Program  in  2009.  The  VJO  Program  is  designed  to 
connect  veterans  to  supports  and  services  to  reduce  and  prevent  re¬ 
incarceration  and  homelessness.2  The  program  relies  on  VJO  specialists 
across  the  United  States  to  forge  and  maintain  relationships  with  jails  and 
municipal  courts  at  the  local  level  (i.e.,  local  criminal  justice  systems). 

Given  the  importance  of  helping  veterans  readjust  to  civilian  life,  you 
asked  us  to  examine  the  management  of  the  VJO  Program.  This  report 
provides  information  on  the  following  questions: 

1 .  How  does  the  Veterans  Justice  Outreach  Program  deliver  services, 
and  what  are  the  number  and  characteristics  of  veterans  served  by 
the  program? 


^AO,  Veterans  Affairs:  Better  Understanding  Needed  to  Enhance  Services  to  Veterans 
Readjusting  to  Civilian  Life,  GAO-14-676  (Washington,  D.C.:  Sept.  10,  2014). 

2The  VJO  Program’s  fiscal  year  2016  budget  is  set  at  about  $31  million,  according  to  VA. 
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2.  To  what  extent  has  VA  used  performance  assessment  to  help 
manage  the  Veterans  Justice  Outreach  Program? 

3.  What  key  challenges,  if  any,  has  VA  identified  and  to  what  extent  has 
the  agency  developed  mitigation  strategies,  as  necessary? 

To  address  our  objectives,  we  reviewed  relevant  federal  laws  and 
regulations,  as  well  as  documents  from  VA’s  central  office,  including  the 
VJO  Program  fiscal  year  2012-2016  strategic  plan  (the  most  recent 
available),  policies,  procedures,  guidance,  and  program  fact  sheets.  We 
did  not  independently  verify  the  actions  described  in  such  documents.  We 
obtained  data  about  the  number  and  characteristics  of  veterans  served  by 
the  VJO  Program  from  VA  for  fiscal  years  2012  through  2015  (the  most 
recent  data  available).  We  determined  that  VA’s  compilation  of  data  about 
veterans  served  by  the  VJO  Program  were  sufficiently  reliable  to  include 
in  our  report.  Specifically,  we  obtained  and  assessed  official 
documentation  such  as  user  guides,  frequently  asked  questions,  and 
disclaimers;  and  we  discussed  our  planned  use  of  the  data  and  any 
limitations  with  VA  officials.  To  assess  the  degree  to  which  VA  uses 
program  performance  assessment — setting  program  goals,  using 
performance  measures,  and  evaluating  programs — to  manage  the  VJO 
Program,  we  compared  VA’s  use  of  performance  assessment  against 
best  practices  for  assessing  program  performance  and  federal  standards 
for  internal  control.3  We  compared  VA’s  efforts  to  identify  challenges  and 
respective  mitigation  strategies  with  criteria  established  in  the  federal 


3We  have  previously  stated  that  performance  goals  and  measures  are  important 
management  tools  that  can  serve  as  leading  practices  for  planning  at  lower  levels  within 
federal  agencies,  such  as  individual  programs  or  initiatives.  For  example,  see  GAO, 
Environmental  Justice:  EPA  Needs  to  Take  Additional  Actions  to  Help  Ensure  Effective 
Implementation,  GAO-12-77  (Washington,  D.C.:  Oct.  6,  2011);  Federal  Prison  System: 
Justice  Could  Better  Measure  Progress  Addressing  Incarceration  Challenges, 
GAO-15-454  (Washington,  D.C.:  July  19,  2015);  DHS  Training:  Improved  Documentation, 
Resource  Tracking,  and  Performance  Measurement  Could  Strengthen  Efforts, 
GAO-14-688  (Washington,  D.C.:  Sept.  10,  2014);  Program  Evaluation:  Strategies  to 
Facilitate  Agencies’  Use  of  Evaluation  in  Program  Management  and  Policy  Making, 
GAO-13-570  (Washington,  D.C.:  June  26,  2013);  Performance  Measurement  and 
Evaluation:  Definitions  and  Relationships,  GAO-1 1-646SP  (Washington,  D.C.:  May  2011); 
Managing  for  Results:  Enhancing  Agency  Use  of  Performance  Information  for 
Management  Decision  Making,  GAO-05-927  (Washington,  D.C.:  Sept.  9,  2005); 
Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3.1 
(Washington,  D.C.:  Nov.  1999). 
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standards  for  internal  control.4  Specifically,  we  chose  to  use  risk 
assessment — one  of  five  key  components  in  standards  for  internal 
control — because  analyzing  risk  provides  the  basis  for  developing 
appropriate  mitigation  strategies. 

We  also  conducted  semi-structured  interviews  with  stakeholders  in  nine 
areas  served  by  VA  medical  centers.  They  are:  (1)  Baltimore,  Maryland; 
(2)  Bedford,  Massachusetts;  (3)  Chicago,  Illinois;  (4)  Fargo,  North  Dakota; 
(5)  Houston,  Texas;  (6)  Orlando,  Florida;  (7)  Salt  Lake  City,  Utah;  (8)  San 
Diego,  California;  and  (9)  Seattle,  Washington.  In  all  areas  we  conducted 
interviews  with  VJO  specialists  and  Veterans  Integrated  Service  Network 
(VISN)  officials  responsible  for  overseeing  the  VA  medical  centers 
included  in  our  review.  In  addition,  for  three  areas — Baltimore,  Maryland; 
Orlando,  Florida;  and  San  Diego,  California5 — we  also  interviewed  VA 
medical  center  officials;  local  criminal  justice  system  stakeholders, 
including  court  coordinators  and  judges,  as  well  as  jail  administrators;  and 
held  discussion  groups  with  a  small  nonprobability  sample  of  veterans 
participating  in  the  VJO  Program.6  We  selected  the  nine  areas  based  on 
the  size  of  their  veteran  populations,  proximity  to  veterans  treatment 
courts,  geographic  diversity,  and  VA  central  office  officials’ 
recommendations.7  The  interviews  we  conducted  in  the  nine  areas  are 
nongeneralizable  but  provided  insights  on  the  challenges  facing  the 
program  and  its  operations.  We  also  interviewed  relevant  VA  central 
office  officials. 


internal  controls  are  the  plans,  methods,  policies,  and  procedures  that  an  entity  uses  to 
fulfill  its  mission,  strategic  plan,  goals,  and  objectives.  GAO/AIMD-OO-21.3.1.  The  most 
recent  version  of  these  standards  was  issued  in  September  2014.  GAO,  Standards  for 
Internal  Control  in  the  Federal  Government,  GAO-14-704G  (Washington,  D.C.:  Sept.  10, 
2014).  These  new  standards  became  effective  October  1,  2015.  We  used  the  previous 
version  of  the  standards  because  we  began  our  audit  work  prior  to  October  1 , 201 5. 

5We  conducted  our  interviews  in  person  in  Baltimore  and  San  Diego.  Due  to  inclement 
weather,  which  prevented  our  travel,  we  conducted  our  Orlando  interviews  via 
teleconference.  While  we  were  not  able  to  observe  a  veterans  treatment  court  in  Orlando, 
we  were  able  to  speak  with  veterans  who  participated  in  the  Orlando  court.  As  an 
alternative,  we  observed  a  veterans  treatment  court  in  Fairfax,  Virginia. 

6VJO  specialists  in  Baltimore,  Orlando,  and  San  Diego  referred  us  to  veterans  who  were 
participating  in  the  Veterans  Justice  Outreach  Program. 

7We  considered  a  VA  medical  center  to  be  located  near  a  veteran  treatment  court  if  the 
court  is  in  the  same  state  and  within  40  driving  miles.  Our  selection  process  included 
some  VA  medical  centers  located  near  a  court  and  some  not  near  a  court. 
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Additional  information  about  our  methodology  and  the  selection  criteria  for 
areas  in  our  review  is  provided  in  appendix  I. 

We  conducted  this  performance  audit  from  February  2015  to  April  2016  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


Description  Of  Local  Local  jurisdictions  (i.e.,  counties  and  municipalities)  administer  criminal 

Criminal  Justice  Systems  courts  and  permit  pretrial  detention  of  defendants  accused  of  serious 

offenses  and  deemed  to  be  dangerous  to  prevent  them  from  committing 
crimes  prior  to  trial,  according  to  the  Bureau  of  Justice  Statistics.8  Many 
local  jurisdictions  have  begun  adopting  alternatives  to  incarceration, 
which  are  intended  to  improve  public  health  and  safety  while  reducing 
costs.  Such  alternatives  generally  represent  a  shift  in  emphasis  away 
from  prosecuting  those  associated  with  lower-level  crimes  toward 
providing  treatment  for  underlying  mental  health  or  substance  abuse 
disorders  and  include  programs  for  individuals  in  jail  or  court  who  screen 
positive  for  mental  illness.  Veterans  treatment  courts  are  one  such 
alternative.  These  courts  are  typically  local  courts  dedicated  to  handling 
criminal  cases  involving  veterans  with  mental  health  or  substance  abuse 
problems. 

According  to  VA,  veterans  treatment  courts  share  several  general 
characteristics  but  vary  in  their  specific  policies  and  procedures  because 
of,  among  other  things,  differences  in  local  jurisdictions  and  criminal 
justice  system  practices.  Veterans  treatment  courts  are  modeled  after 
adult  drug  courts,  which  are  specialized  courts  that  target  criminal 
offenders  who  have  drug  addiction  and  dependency  problems.  As  in  the 
case  of  adult  drug  courts,  judges  preside  over  veterans  treatment  court 


8Offenders  sentenced  to  incarceration  usually  serve  time  in  a  local  jail  or  a  state  prison. 
Offenders  sentenced  to  less  than  1  year  generally  go  to  jail;  those  sentenced  to  more  than 
1  year  go  to  prison,  according  to  the  Bureau  of  Justice  Statistics. 
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proceedings  and  monitor  veterans’  progress  with  treatment  in 
collaboration  with  a  team  that  usually  includes  a  court  coordinator, 
prosecutor,  public  defender,  and  probation  officer.  Additionally,  the  team 
includes  a  VJO  specialist.9  Veterans  treatment  courts  vary  in  terms  of 
criteria  for  taking  a  case,  such  as  the  types  or  levels  of  criminal  offenses. 


Involvement  of  Veterans  in 
Local  Criminal  Justice 
Systems 


As  was  the  case  with  past  generations  of  veterans,  the  transition  from 
military  life  to  civilian  life  can  be  challenging  for  Post-9/11  veterans.  Most 
veterans  are  not  involved  with  local  criminal  justice  systems,10  but  some 
veterans — particularly  if  their  mental  health,  family  readjustment,  or  other 
needs  remain  unmet — may  become  justice-involved.  The  Bureau  of 
Justice  Statistics  reported  that  about  7  percent  (about  50,000)  of  the  total 
population  of  inmates  in  jail  between  February  201 1  and  May  2012  were 
veterans.11  According  to  the  Bureau  of  Justice  Statistics  report,  this 
estimate  represents  a  25  percent  decrease  from  the  number  of  veterans 
in  jail  in  2004.  Moreover,  veterans  were  incarcerated  in  jails  at  lower  rates 
than  nonveterans  between  February  201 1  and  May  2012,  according  to 
the  Bureau  of  Justice  Statistics. 


While  many  veterans  who  served  in  the  military  have  successfully 
readjusted  to  civilian  life  with  minimal  difficulties,  researchers  and 
policymakers  have  identified  concerns  about  how  the  experience  of  Post- 
9/1  1  servicemembers  may  affect  the  incarceration  rates  among  these 
veterans.  According  to  VA,  military  experience  (particularly  combat)  has 


9Veterans  treatment  courts  and  related  programs  rely  heavily  on  VA  for  services  and 
benefits  provided  to  veterans. 

1  f) 

Currently  there  are  approximately  22  million  veterans  in  the  United  States. 

^See,  J.  Bronson;  A.  Carson;  M.  Noonan;  and  M.  Berzofsky  (2015).  Veterans  in  Prison 
and  Jail,  2011  -  12.  Bureau  of  Justice  Statistics.  These  findings  are  based  on  the 
February  201 1  through  May  2012  fielding  of  the  National  Inmate  Survey  (NIS-3).  The  NIS- 
3  surveyed  veterans  age  1 8  and  older  identified  in  a  survey  of  inmates  in  a  sample  of  233 
state  and  federal  prisons,  358  local  jails,  and  15  special  facilities  (military,  Indian  country, 
and  Immigration  and  Customs  Enforcement  (ICE))  nationwide.  However,  these  findings 
underreport  the  number  of  veterans  in  the  local  criminal  justice  system.  For  example,  the 
survey  does  not  provide  separate  data  on  the  number  of  veterans  under  direct  community 
supervision,  such  as  those  on  probation,  or  those  involved  in  specialty  courts,  such  as 
veterans  treatment  courts.  Further,  the  survey  relies  on  inmates  to  self-report  their  veteran 
status  and  according  to  several  researchers,  veterans  may  be  reluctant  to  disclose  their 
veteran  status  (see  W.  Brown,  R.  Stanulis,  B.  Theis,  J.  Farnsworth.,  and  D.  Daniels 
(2013),  The  Perfect  Storm:  Veterans,  culture  and  the  criminal  justice  system.  Justice 
Policy  Journal,  vol.  10,  no.  2  (Fall)). 
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been  an  underlying  factor  in  behavior  that  prompts  a  response  from  law 
enforcement,  such  as  domestic  conflicts.  During  the  last  14  years  of  U.S. 
military  operations,  many  servicemembers  have  experienced  numerous 
deployments,  which  can  increase  the  risk  of  developing  post-traumatic 
stress  disorder  (PTSD)  and  traumatic  brain  injury.  According  to  VA,  a 
strong  relationship  exists  between  PTSD  and  substance  abuse.12 
Research  has  demonstrated  that  justice-involved  veterans  have  high 
rates  of  mental  illness,  substance  abuse,  homelessness,  and  other  health 
issues.  At  the  same  time,  some  veterans  may  be  unwilling  or  unable  to 
access  the  supports  and  services  they  need.  For  example,  we  have 
previously  reported  that  some  veterans  do  not  seek  mental  health 
treatment  due  to  concerns  about  negative  career  outcomes,  lack  of 
understanding  or  awareness  of  treatment,  and  logistical  challenges  to 
accessing  care.13  Left  unaddressed,  a  combination  of  homelessness, 
unemployment,  mental-health,  or  substance  abuse  issues  can  place 
veterans  experiencing  a  difficult  transition  at  higher  risk  of  committing  a 
crime. 


Description  of  the 
Veterans  Justice  Outreach 
Program 


VA  initiated  the  VJO  Program  in  2009. 14  The  mission  of  the  program  is  to 
reduce  and  prevent  criminal  justice  recidivism  and  homelessness  among 
veterans  by  linking  justice-involved  veterans  with  appropriate  supports 
and  services.  According  to  VA,  incarceration  is  a  strong  predictor  of 
veteran  homelessness,  and  recidivism  can  limit  VA’s  ability  to  provide 
continuous  care  for  mental  health  and  other  issues. 


The  VJO  Program’s  fiscal  year  2012-2016  strategic  plan  contains  five 
broad  strategic  goals  for  the  program  as  well  as  a  number  of  related 
objectives  (see  table  1). 


12Department  of  Veterans  Affairs,  National  Center  for  PTSD,  PTSD  and  Substance  Abuse 
in  Veterans,  accessed  Feb.  10,  2016, 

http://www.ptsd.va.gov/public/problems/ptsd_substance_abuse_veterans.asp. 

13GAO,  VA  Mental  Health:  Number  of  Veterans  Receiving  Care,  Barriers  Faced,  and 
Efforts  to  Increase  Access,  GAO-12-12  (Washington,  D.C.:  Oct.  14,  2011). 

1 4 

According  to  a  program  official,  VA  has  associated  the  VJO  Program  with  38  U.S.C.  § 
2023,  and  the  language  in  38  U.S.C.  §  2022  is  believed  to  be  broad  enough  to  authorize 
much  of  the  program’s  operations.  A  bill  has  been  introduced  in  the  Senate  that  would 
formally  require  VA  to  carry  out  the  VJO  Program.  S.  2120,  1 14th  Cong.  (2015). 
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Table  1:  Veterans  Justice  Outreach  (VJO)  Program  Strategic  Goals  and  Related  Objectives 

Strategic  Goals 

Strategic  Objectives 

Goal  1:  Increase  the  identification  of  justice-involved  veterans 
and  promote  their  engagement  with  VA. 

Objective  1:  Collaborate  with  the  justice  system  and  other 
community  entities  to  develop  opportunities  for  identifying  veterans. 

Objective  2:  Conduct  outreach  to  justice-involved  veterans  to  link 
them  to  the  appropriate  VA  and  community  services. 

Goal  2:  Build  VJO  capacity  to  serve  justice-involved  veterans 
through  a  comprehensive  workforce  development  program  that 
attracts  and  retains  high-quality  staff,  supports  their 
professional  development  efforts,  and  supports  succession 
planning. 

Objective  1:  Attract  and  retain  high-quality  staff. 

Objective  2:  Support  professional  development. 

Objective  3:  Initiate  succession  planning  strategies. 

Goal  3:  Match  justice-involved  veterans  with  the  medical, 
psychiatric,  vocational,  and  social  services  that  will  improve 
health  and  optimize  successful  community  integration  and 
safety. 

Objective  1:  Assess  justice-involved  veterans  for  biopsychosocial 
needs  and  problems.3 

Objective  2:  Develop  VA-community  partnerships  that  address  the 
needs  of  justice-involved  veterans. 

Goal  4:  Optimize  community  integration  for  justice-involved 
veterans  by  removing  attitudinal  barriers  through  improving 
stakeholder  understanding  of  the  population. b 

Objective  1:  Educate  stakeholders  on  veteran’s  available  treatment 
options  and  potential  for  community  integration  after  becoming 
involved  in  the  criminal  justice  system,  particularly  those  with  mental 
illness  and  those  returning  after  military  deployment. 

Goal  5:  Develop  systems  for  program  evaluation  and 
dissemination  of  knowledge  on  justice-involved  veterans. 

Objective  1:  Ensure  capacity  for  program  data  collection  and 
analysis,  promotion  of  research  on  justice-involved  veterans,  and 
knowledge  development  and  dissemination. 

Objective  2:  Develop  national  data  collection,  management,  and 
evaluation  systems. 

Objective  3:  Develop  a  VJO  research  initiative  to  promote  timely 
identification  and  successful  community  integration. 

Source:  VA’s  fiscal  year  2012-2016  Veterans  Justice  Outreach  Program  Strategic  Plan.  |  GAO-16-393 

aBiopsychosocial  assessments  include  the  collection  of  medical,  psychiatric,  social,  developmental, 
legal,  and  abuse  histories  along  with  other  key  information. 

bln  the  program’s  fiscal  year  2012-2016  strategic  plan,  VA  identifies  a  number  of  internal 
stakeholders,  such  as  VA  homeless  and  mental  health  program  officials;  as  well  as  external 
stakeholders,  such  as  representatives  from  the  National  Association  of  Drug  Court  Professionals  and 
the  Association  of  State  Correctional  Administrators. 


Importance  Of  Program  Evaluative  information  helps  the  executive  branch  and  congressional 
Performance  Assessment  committees  make  decisions  about  the  programs  they  oversee;  that  is, 

evaluative  information  tells  them  whether  and  why  a  program  is  working 
well  or  not.  We  have  previously  reported  that  an  important  system  to 
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obtain  such  information  is  through  program  performance  assessment.15 
(See  fig.  1 .)  A  program  performance  assessment  system  is  an  important 
component  of  effective  program  management  and  contains  three  key 
elements: 


1 .  Program  goals  communicate  what  the  agency  proposes  to 
accomplish  and  allow  agencies  to  assess  or  demonstrate  the  degree 
to  which  those  desired  results  were  achieved. 

•  Strategic  goals  and  related  objectives  are  long-term  goals  that  set 
a  general  direction  for  a  program’s  efforts. 

•  Performance  goals  are  the  specific  results  an  agency  expects  its 
program  to  achieve  in  the  near  term. 

2.  Performance  measures  are  concrete,  objective,  observable 
conditions  that  permit  the  assessment  of  progress  made  toward  the 
agency’s  goals.  Performance  measures  show  the  progress  the 
agency  is  making  in  achieving  performance  goals. 

3.  Program  evaluations  are  individual  systematic  studies  using 
performance  measures  and  other  information  to  answer  specific 
questions  about  how  well  a  program  is  meeting  its  objectives. 


Figure  1:  Key  Elements  of  Program  Performance  Assessment  Systems 


Set  strategic  goals 


Source:  GAO.  |  GAO-16-393 


Set  performance  goals 
for  expected  results 


Use  performance  measures 
Performance  measures  allow  organizations 
to  track  progress  in  achieving  their  goals  by 
comparing  actual  performance  against  planned 
or  expected  results,  including  identifying  any  gaps. 


'<9> 


Conduct  program  evaluations 

These  individual,  systematic  studies  are 
conducted  periodically  or  on  an  ad  hoc  basis 
to  assess  how  well  a  program  is  working. 


Performance 

Assessment 

System 


15GAO,  Program  Evaluation:  Strategies  to  Facilitate  Agencies’  Use  of  Evaluation  in 
Program  Management  and  Policy  Making,  GAO-13-570  (Washington,  D.C.:  June  26, 
2013);  Performance  Measurement  and  Evaluation:  Definitions  and  Relationships, 

GAO-1 1-646SP  (Washington,  D.C.:  May  2011);  and  Managing  for  Results:  Enhancing 
Agency  Use  of  Performance  Information  for  Management  Decision  Making,  GAO-05-927 
(Washington,  D.C.:  Sept.  9,  2005). 
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The  VJO  Program 
Operates  Through  VA 
Medical  Centers  and 
Has  Linked  Many 
Post-9/11  Veterans  to 
Supports  and 
Services 


VJO  Program  Operates 
Through  VA  Medical 
Centers  to  Respond  to 
Local  Needs 


The  VJO  Program  operates  through  VA  medical  centers.  VA’s  central 
office  provides  each  VA  medical  center  the  flexibility  to  determine  how  to 
best  respond  to  the  needs  of  justice-involved  veterans  within  the  local 
community.  More  specifically,  VA  established  broad  program  parameters 
that  allow  VA  medical  center  officials  to  set  VJO  specialists’  activities  to 
meet  the  needs  of  veterans  in  local  criminal  justice  systems.  VA  issued 
guidelines  on  the  operations  of  the  VJO  Program  through  a  series  of 
memorandums  to  program  officials.16  According  to  VA,  the  program  is  to 
serve  veterans  as  they  interact  with  the  local  criminal  justice  system  at 
multiple  points  and  settings — from  initial  contact  with  local  law 
enforcement  to  release  from  jail.  VA  guidelines  define  a  “justice-involved 
veteran”  as  a  veteran  who  is  (1)  arrested  by  local  law  enforcement  who 
can  be  appropriately  diverted  from  arrest  into  treatment;  (2)  incarcerated 
in  a  local  jail,  and  who  either  has  a  pending  trial  or  is  serving  a  sentence 
after  a  conviction;  or  (3)  involved  in  adjudication  or  monitoring  by  a  court. 
(See  fig.  2.) 


16See,  for  example,  Under  Secretary  for  Health,  Information  and  Recommendation  for 
Services  Provided  by  VHA  Facilities  to  Veterans  in  the  Criminal  Justice  System, 
Information  Letter  10-2009-005,  April  30,  2009;  and  Acting  Deputy  Under  Secretary  for 
Health,  Access  to  VA  services  for  re-entry  and  justice-involved  veterans,  Information 
Letter,  July  20,  2009.  Also,  during  the  course  of  our  audit,  VA  central  office  officials  were 
in  the  process  of  drafting  a  handbook  for  Veterans  Justice  Programs — which  include  VJO 
and  Health  Care  for  Re-entry  Veterans  programs — that  will  establish  policies  and 
procedures  for  implementation  of  these  programs,  such  as  monitoring. 
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Figure  2:  Overview  of  Steps  in  the  Local  Criminal  Justice  Process  in  which  Veterans  Justice  Outreach  (VJO)  Program 
Interacts  with  Veterans 


Arrest 

and  booking 


Pre-trial 

detention 


Jail  sentencing 
detention 


Probation 


Veterans 
treatment  court 


Source:  Department  of  Veterans  Affairs'  Veterans  Justice  Outreach  (VJO)  Program  2012-2016  Strategic  Plan  |  GAO-16-393 


The  Veterans  Health  Administration  (VHA),  which  operates  VA’s  health 
care  system,  administers  the  VJO  Program.  VA  policy  requires  each  of 
the  167  VA  medical  centers  around  the  country  to  provide  outreach  to 
justice-involved  veterans.  VA  prioritizes  veterans  charged  with  nonviolent 
crimes,  but  VA  must  consider  a  veteran’s  current  legal  circumstances — 
and  not  legal  history  alone — to  determine  whether  the  program  can  meet 
the  individual  veteran’s  needs  while  maintaining  safety,  according  to  VA 
policy.  The  program  does  not  provide  legal  representation,  nor  does  it 
accept  legal  custody  of  a  veteran. 

Specifically,  VHA  staff  in  Veterans  Integrated  Service  Networks  (VISN) 
help  oversee  the  VJO  Program  and  provide  technical  assistance  to  VJO 
specialists.  Further,  VA’s  central  office,  its  VISNs,  and  VA  medical 
centers  manage  different  aspects  of  program  operations.  VA  central  office 
officials  directed  each  medical  center  to  have  at  least  one  full-time  VJO 
specialist  position,  most  of  which  are  funded  by  central  office  funds, 
according  to  VA.17  Some  VA  medical  centers  employ  one  VJO  specialist 
position  while  others  have  multiple  positions.  VA  central  office  officials 
also  train  new  VJO  specialists  and  manage  national  data  collection 
efforts.  VJO  specialists  report  directly  to  VA  medical  center  officials, 
typically  in  the  homelessness  prevention  or  mental  health  services. 


17VA  central  office  officials  we  interviewed  said  that  although  most  of  the  VJO  specialist 
positions  are  funded  by  central  office  funds,  a  few  of  the  VA  medical  centers  have  chosen 
to  fund  additional  VJO  specialist  positions. 
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According  to  VA  central  office  officials,  as  of  September  2015,  VA 
employed  261  full-time  VJO  specialists,  who  are  mostly  social  workers, 
and  seven  other  VJO  specialists  who  play  a  hybrid  role  between  the  VJO 
and  the  Health  Care  for  Re-entry  Veterans  programs.18 

To  accomplish  the  program  mission  of  serving  justice-involved  veterans, 
VA  tasks  VJO  specialists  to  perform  three  core  functions:  identify,  assess, 
and  link  justice-involved  veterans  to  appropriate  supports  and  services. 

Identification  To  identify  veterans,  VJO  specialists  conduct  outreach  to  justice-involved 

veterans  and  officials  in  local  criminal  justice  systems.  Outreach  activities 
include,  for  example,  providing  information  about  the  availability  of  VA 
supports  and  services,  developing  relationships  with  municipal  and 
county  jail  officials  and  court  officials,  as  well  as  obtaining  appropriate 
security  clearances.  VJO  specialists  in  seven  of  the  nine  areas  we 
selected  for  interviews  stated  that  once  VJO  specialists  have  developed  a 
professional  relationship  with  criminal  justice  system  officials,  they 
primarily  identify  justice-involved  veterans  by  obtaining  a  list  from  local  jail 
administrators  of  self-identified  veterans  in  their  jails.  Other  veterans  are 
identified  by  veterans  treatment  court  officials,  according  to  VA  central 
office  officials  and  staff  we  interviewed.  However,  some  incarcerated 
veterans  may  be  reluctant  to  disclose  their  veteran  status.19  Thus,  jail 
administrators  may  have  more  incarcerated  veterans  in  their  jails  than 
they  are  able  to  identify  and  report  to  VA.  To  address  the  potential  for 
underreporting  of  incarcerated  veterans,  VA  developed  the  Veterans  Re¬ 
entry  Search  Service  for  administrators  of  jails  and  state  and  federal 
prisons,  and  VA  central  office  officials  we  interviewed  said  that  the  system 


1  ft 

The  Health  Care  for  Re-entry  Veterans  program  was  established  in  2006.  It  serves 
veterans  in  prison  (not  jails)  to  plan  their  transition  from  incarceration  back  into  the 
community,  and  connects  veterans  being  released  from  prison  with  VA  services,  such  as 
housing  assistance  and  vocational  counseling. 

1 Q 

Some  of  the  reasons  that  inmates  do  not  self-identify  include  being  ashamed  of  their 
breakdown  in  military  discipline,  embarrassed  over  being  arrested,  and  fear  of  losing  VA 
benefits  for  themselves  or  their  families.  See,  for  example,  William  B.  Brown,  Robert 
Stanulis,  Bryan  Theis,  Jordan  Farnsworth  and  David  Daniels,  “The  Perfect  Storm: 
Veterans,  culture  and  the  criminal  justice  system,”  Justice  Policy  Journal,  vol.  10,  no.  2 
(Fall  2013). 
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helps  jail  administrators  identify  incarcerated  veterans  by  comparing  the 

Experience  of  veterans  who  Have  names  of  inmates  with  VA’s  list  of  veterans.20 

Participated  in  the  VJO  Program 

All  of  the  veterans  participating  in  our  group 
discussions  were  appreciative  of  the  VJO 
Program.  At  one  location,  veterans  said  that 
the  VJO  specialists  understood  their  needs, 
educated  them  about  available  VA  supports 
and  services,  assisted  them  in  determining 
their  eligibility,  and  helped  them  obtain  VA 
supports  and  services. 

An  example  from  a  VJO  Specialist 

To  illustrate  the  effect  of  the  program  on  one 
veteran’s  life,  VJO  specialists  we  interviewed 
in  one  of  our  selected  areas  provided  the 
following  anecdote.  A  homeless  veteran — who 
had  served  in  Operation  Enduring  Freedom — 
was  arrested  for  trespassing  for  sleeping 
behind  a  VA  building.  The  veteran  had 
substance  abuse  issues.  When  he  appeared 
before  a  veterans  treatment  court,  he  met  a 
VJO  specialist  who  assessed  his  needs  and 
assisted  him  in  enrolling  in  VA  housing  and 
substance  abuse  treatment.  When  the  veteran 
was  stable,  the  VJO  specialist  was  able  to  link 
him  to  permanent  housing  and  VA  vocational 
training.  After  several  years  of  treatment  and 
assistance,  the  veteran  was  hired  by  VA  and 
purchased  a  home  with  a  VA  loan. 

Source:  Veterans  participating  in  Veterans  Justice  Outreach 
Program  and  VJO  specialists  interviewed  by  GAO.  | 

GAO-16-393 


Assessment  After  they  identify  justice-involved  veterans,  VJO  specialists  determine 

the  veterans’  treatment  needs  by  assessing  their  mental  health,  social 
well-being,  appearance,  and  attitude.21  VJO  specialists  also  collect 
information  on  employment  history,  current  housing  situation,  military 
service,  and  discharge  status.  According  to  VA  staff,  upon  completing  the 
assessment,  VJO  specialists  develop  a  treatment  plan  to  meet  the 
veteran’s  needs.  A  treatment  plan  typically  includes  recommendations  for 


on 

Veterans  Re-entry  Search  Service  is  an  online  tool  that  enables  jail  administrators  to 
identify  veterans  among  their  incarcerated  population.  Jail  administrators  can  upload  a  list 
of  their  current  inmates  into  VA’s  system,  which  is  checked  against  the  list  of  veterans 
found  in  the  VA-DoD  Identity  Repository.  Jail  administrators  can  use  this  information  to 
provide  VJO  specialists  with  a  list  of  veterans  in  their  jails. 

21According  to  VJO  specialists  in  five  of  the  nine  areas  we  selected  for  interviews,  the 
assessment  protocol  to  evaluate  a  veteran  is  generally  the  same  for  all  justice-involved 
veterans. 
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Linkage 


medical  or  mental  health  services,  housing,  or  other  services,  according 
to  VJO  specialists  in  five  of  the  nine  selected  areas.22 

After  they  identify  and  assess  justice-involved  veterans,  VJO  specialists 
link  veterans  to  VA  or  community  supports  and  services.23  VJO 
specialists  do  not  directly  provide  treatment  to  justice-involved  veterans. 
VA  staff  said  once  VJO  specialists  refer  and  link  veterans  to  the 
appropriate  supports  and  services,  they  perform  follow-up  visits  with 
veterans  to  ensure  they  are  receiving  them.  For  example,  VJO  specialists 
may  assist  veterans  to  find  adequate  housing  or  with  any  transportation 
issues.  Some  veterans  in  jails  may  receive  one  follow-up  visit  while  others 
may  receive  more,  depending  on  their  needs  according  to  a  VJO 
specialist.  In  contrast,  VJO  specialists  work  with  veterans  who  participate 
in  veterans  treatment  courts  for  1  to  2  years,  depending  on  the  local 
criminal  justice  system  and  on  the  amount  of  time  a  veteran  is  required  to 
participate  in  the  court  program. 

VJO  specialists  refer  and  link  veterans  to  the  VHA  for  health  care,  mental 
health,  substance  abuse  treatment,  or  housing  services  where  the 
overseeing  staff  determine  the  type  of  treatment  and  services  provided.24 
VJO  specialists  also  link  veterans  to  the  Veterans  Benefits  Administration 
for  disability  compensation,  pension  benefits,  or  vocational  rehabilitation 
to  be  determined  by  overseeing  staff  based  on  a  veteran’s  eligibility, 
according  to  VA  (see  fig.  3).  In  addition,  VJO  specialists  link  veterans  to 
community  service  providers  when  the  VA-provided  treatment  is  too  far 
away  for  a  veteran  to  participate,  or  when  a  veteran  is  not  eligible  for  VA 
health  care  services  or  benefits  according  to  VA. 


22VJO  specialists  enter  the  veteran’s  information  into  VA  Homeless  Operations 
Management  and  Evaluation  System  and  the  Computerized  Patient  Record  System.  Once 
the  veteran’s  information  is  entered  into  VA’s  databases,  it  is  accessible  to  other  VA 
clinicians  treating  the  veteran,  according  to  VA  staff. 

oo 

Veterans  not  participating  in  court  can  decide  whether  or  not  to  accept  VA  supports  and 
services  since  the  program  is  voluntary. 

24UnderVHA’s  Homelessness  Division,  the  Housing  and  Urban  Development-Veterans 
Affairs  Supportive  Housing  Program  provides  permanent  housing  for  eligible  homeless 
veterans  and  their  families. 
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Figure  3:  Veterans  Justice  Outreach  Specialists  Link  Justice-Involved  Veterans  to 
Services 
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Within  these  broad  program  parameters,  a  VA  medical  center  determines 
the  type  and  amount  of  investment  it  makes  in  serving  justice-involved 
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veterans.25  VJO  specialists’  activities  vary  based  on  whether  they  are 
working  in  courts,  jails,  or  both.  For  example: 

•  Specialty  courts:  VJO  specialists  provide  services  to  veterans  in  a 
variety  of  specialty  courts,  such  as  veterans  treatment  courts,  drug 
courts,  and  mental  health  courts.26  However,  VJO  specialists  work 
most  often  with  veterans  treatment  courts  compared  to  other  specialty 
courts,  according  to  VA  central  office  officials. 

•  Local  criminal  justice  systems:  In  local  criminal  justice  systems 
without  established  veteran-focused  programs,  VJO  specialists  spend 
significant  time  attempting  to  develop  or  plan  these  types  of  programs, 
according  to  VISN  officials.  These  activities  involve  but  are  not  limited 
to  planning  veterans  treatment  court  (or  other  alternative  court) 
programs  and  negotiating  the  terms  of  access  to  conduct  outreach  in 
jail  facilities.  In  other  areas,  VJO  specialists  work  with  local  criminal 
justice  officials  who  may  be  unaware  of  veteran-focused  programs.  In 
such  cases,  VJO  specialists  focus  their  work  on  educating  local  law 
enforcement  about  VA  resources  available  to  veterans  in  crisis  and 
working  with  veterans  in  jails,  according  to  two  VISN  officials  with 
whom  we  spoke. 

•  Jail  administrators:  VJO  specialists  in  two  of  the  nine  areas  we 
selected  for  interviews  worked  with  local  jail  administrators  to  set  up  a 
program  that  provides  veteran-specific  housing  units.  These  housing 
units  are  a  designated  block  of  jail  cells  only  for  veterans.  Within  these 
units,  local  community  service  organizations  provide  incarcerated 
veterans  with  treatment  and  services  while  they  are  in  jail,  and  VJO 
specialists  assess  and  develop  community  re-entry  plans  for  housing, 
VA  treatment,  or  other  necessary  services.  In  another  area  we  visited, 
VJO  specialists  developed  a  program  to  help  “high-risk”  veterans 
released  from  jail.  VJO  specialists  used  information  from  the  veterans’ 


25The  associated  costs  of  providing  treatment  and  benefits  to  veterans  involved  with  local 
criminal  justice  systems  are  borne  primarily  by  the  federal  government,  according  to  VA. 
Further,  veterans  treatment  courts  would  not  likely  be  able  to  fully  function  and  proliferate 
without  VA,  as  the  agency  provides  viable  access  to  VA  services,  according  to  VA’s  2013 
inventory  of  VA  involvement  in  veterans  treatment  courts. 

26The  types  of  drug  courts  include  but  are  not  limited  to  adult  drug  courts  and  veterans 
drug  courts  for  offenders  with  mental  health  and  substance  addiction  issues.  GAO,  Adult 
Drug  Courts:  Studies  Show  Courts  Reduce  Recidivism,  but  DOJ  Could  Enhance  Future 
Performance  Measure  Revision  Efforts,  GAO-12-53  (Washington,  D.C.:  Dec.  9,  2011). 
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clinical  assessment  to  determine  which  veterans  were  high-risk  for  re¬ 
offending,  dropping  out  of  treatment,  or  becoming  homeless.  One 
VJO  specialist  at  that  VA  medical  center  said  the  program  serves 
these  veterans  by  providing  weekly  follow-up  services  within  the  first 
month  of  their  release  from  jail,  and  offers  regular  follow-up  services 
and  support  after  the  first  month,  based  on  the  individual  veteran’s 
needs.27 


In  fiscal  year  2015,  the 
VJO  Program  Served 
about  46,500  Veterans, 
Many  of  Whom  Served  in 
Post-9/11  Conflicts  and 
Reported  Substance 
Abuse  Problems 


In  fiscal  year  201 5,  the  VJO  Program  served  about  46,500  justice- 
involved  veterans,  and  the  program  has  experienced  steady  growth  in  the 
number  of  veterans  served,  according  to  data  from  VA.  During  fiscal 
years  2012  through  2015,  the  number  of  justice-involved  veterans 
annually  served  by  the  VJO  Program  increased  from  about  27,000  to 
46,500,  a  72  percent  increase  (see  fig.  4). 


Figure  4:  Number  of  Veterans  Served  Annually  by  the  Veterans  Justice  Outreach 
Program,  Fiscal  Years  2012  through  2015 
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Source:  GAO  analysis  of  Department  of  Veterans  Affairs  data.  |  GAO-16-393 


27VJO  specialists  address  veterans’  issues  such  as  obtaining  and  maintaining  housing, 
maintaining  treatment  compliance,  applying  for  VA  benefits,  accessing  VA  healthcare 
services,  and  obtaining  gainful  employment. 
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aThe  number  of  justice-involved  veterans  represents  veterans  who  were  assessed  by  Veterans 
Justice  Outreach  specialists  and  agreed  to  enter  the  program. 


Justice-involved  veterans  who  were  served  by  the  VJO  Program  in  fiscal 

year  2015  had  the  following  characteristics:28 

•  Most  were  young  men.  Nearly  95  percent  of  the  veterans  served  by 
the  program  were  male  and  52  percent  of  justice-involved  veterans 
were  between  the  ages  of  18  and  44. 

•  Many  reported  not  working  full-time  during  the  past  3  years. 

Approximately  40  percent  of  the  veterans  served  by  the  program 
reported  working  part  time,  working  irregularly,  or  having  been 
unemployed  during  the  past  3  years.  Another  one-third  (33  percent) 
reported  being  retired  or  disabled. 

•  Almost  three-fourths  reported  serving  during  a  U.S.  military 
intervention.  Of  veterans  who  received  services  through  the  VJO 
Program  in  fiscal  year  2015,  73  percent  had  served  during  a  military 
intervention  (see  fig. 5). 29 


oo 

As  part  of  the  assessment,  VJO  specialists  ask  veterans  to  report  their  demographic 
information,  employment,  military  history,  living  situation,  and  other  personal  information. 

pq 

According  to  VA,  a  veteran  is  considered  to  have  served  in  a  military  intervention  if 
he/she  served  during  and  within  the  geographic  proximity  of  a  military  conflict. 
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Figure  5:  Percent  of  Justice-Involved  Veterans  Who  Served  During  Military 
Interventions,  Fiscal  Year  2015 
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Source:  GAO  analysis  of  Department  of  Veterans  Affairs  data.  |  GAO-16-393 


aNon-military  interventions  include  veterans  who  served  during  peacetime. 

bOther  peacekeeping  or  military  interventions  include  veterans  who  served  during  World  War  II  and 
during  interventions  in  Korea,  Lebanon,  Panama,  or  Kosovo. 


•  Two-thirds  reported  mental  health  problems  and  the  vast 
majority  had  substance  abuse  problems.  During  the  clinical 
assessment  performed  by  VJO  specialists,  about  two-thirds  (68 
percent)  of  the  veterans  reported  one  or  more  mental  health 
problems,  and  69  percent  had  substance  abuse  problems. 

•  Public  order  offenses  were  the  most  common  type  of  reported 
criminal  charges.30  The  types  of  criminal  offenses  that  veterans  were 
charged  with  during  fiscal  year  2015  included,  but  were  not  limited  to, 
public  order  offenses  (33  percent),  drug  offenses  (22  percent), 
property  offenses  (16  percent),  and  probation  violations  (12  percent). 


on 

According  to  VA,  public  order  offenses  include  weapons  offenses,  prostitution,  public 
intoxication,  disorderly  conduct,  and  driving  while  intoxicated;  drug  offenses  include 
possession  and  trafficking;  and  property  offenses  include  burglary,  breaking  and  entering, 
larceny,  motor  vehicle  theft,  fraud,  stolen  property,  arson,  shoplifting,  and  vandalism. 
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•  More  than  40  percent  reported  either  being  homeless,  losing  a 
home,  or  living  in  an  unstable  housing  environment  (see  fig.  6). 
VJO  specialists  we  interviewed  in  two  of  the  nine  areas  we  selected 
said  that  some  veterans  become  homeless  once  they  leave  jail 
because  they  are  unable  to  pay  their  rent  while  incarcerated. 


Figure  6:  Current  Housing  Situation  of  Justice-Involved  Veterans,  by  Percentage, 
Fiscal  Year  2015 

-  3%  Unknown3 


Source:  GAO  analysis  of  Department  of  Veterans  Affairs  data.  |  GAO-16-393 

“Unknown”  refers  to  a  veteran  who  did  not  answer  the  question. 

b“Stable  housing”  refers  to  a  veteran  who  is  not  experiencing  homelessness  and  who  is  not  at  risk  of 
becoming  homeless. 

“Homeless”  refers  to  veterans  who  live  in  places  not  designed  for  habitation  such  as  a  car,  park,  or 
abandoned  building. 

d“lmminent  risk  of  losing  home"  refers  to  a  veteran  who  will  lose  his  or  her  residence  within  14  days. 

“Unstable  housing”  refers  to  a  veteran  who  does  not  have  resources  to  prevent  him  or  her  from 
moving  into  a  shelter. 

Additional  information  about  the  characteristics  of  veterans  in  the  VJO 
Program  is  provided  in  appendix  II. 
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VA  Has  Taken  Steps 
toward  Partially 
Incorporating  Key 
Elements  of  a 
Performance 
Assessment  System 


VA  Is  at  the  Beginning 
Stage  of  Planning  VJO 
Program  Evaluations 


VA  has  taken  some  steps  toward  evaluating  the  VJO  Program  via  a 
longitudinal  program  evaluation  that  examines  the  extent  to  which 
veterans  are  being  linked  to  services  for  which  they  are  referred.31  VA 
planned  to  obtain  information  through  interviews  with  1,500  veterans. 
However,  after  the  contractor  submitted  an  interim  report  that  detailed 
problems  it  encountered  recruiting  veterans  to  interview  as  part  of  the 
study,  VA  central  office  officials  terminated  the  contract.  As  an  alternative, 
VA  plans  to  use  administrative  data  to  complete  this  evaluation.  However, 
at  the  time  of  our  review,  VA  central  office  officials  were  in  the  process  of 
determining  how  to  complete  such  a  re-tooled  evaluation. 

VA  also  developed  a  research  agenda  to  evaluate  the  outcomes  for 
veterans  served  by  the  VJO  Program.  For  example,  one  research  topic  is 
to  identify  outcomes  for  veterans  who  participate  in  veterans  treatment 
courts.  In  addition,  officials  said  they  plan  to  evaluate  the  extent  to  which 
veterans  served  by  the  VJO  Program  avoided  incarceration  and 
homelessness. 


VA  Has  Not  Established 
Performance  Goals  or 
Performance  Measures 


Although  VA  has  developed  five  broad  strategic  goals  for  the  VJO 
Program  and  taken  steps  to  evaluate  it,  VA  has  not  fully  measured 
progress  toward  any  of  the  strategic  goals  because  it  has  developed 
neither  performance  goals  nor  performance  measures,  contrary  to  leading 
practices  for  managing  programs.  Performance  measures  focus  on 
whether  a  program  has  achieved  measurable  standards.  They  allow 


31  Program  evaluations  are  individual  systematic  studies  conducted  periodically  or  on  an 
ad  hoc  basis  to  assess  how  well  a  program  is  working.  They  are  one  component  of  a 
performance  assessment  system,  which  also  includes  strategic  goals,  performance  goals, 
and  performance  measures.  See  GAO,  Performance  Measurement  and  Evaluation: 
Definitions  and  Relationships,  GAO-1 1-646SP  (Washington,  D.C.:  May  2011). 
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agencies  to  monitor  and  report  program  accomplishments  on  an  ongoing 
basis.32 

VA  collects  ongoing  information  about  program  operations  and  veterans 
served  by  the  program,  but  this  information  does  not  help  VA  measure  its 
progress  toward  accomplishing  any  of  the  five  goals  outlined  in  its 
strategic  plan.  According  to  VA  central  office  officials,  VA  collects  some 
information,  such  as  (1)  the  number  of  VJO  specialists  and  vacancies  at 
each  VA  medical  center;  (2)  VJO  specialists’  workload,  including  the 
number  of  veterans  that  VJO  specialists  serve  and  their  nonclinical 
activities,  such  as  the  number  of  trainings  they  conduct;  and  (3) 
information  on  veterans  served  in  the  program,  such  as  demographic 
characteristics  and  medical  histories.  This  information  is  useful  for  some 
aspects  of  program  management.  For  example,  VA  central  office  officials 
told  us  that  supervisors  need  information  about  the  clinical  and 
administrative  workloads  of  VJO  specialists  to  assess  the  need  for 
additional  staff  and  to  monitor  productivity.  However,  VA  does  not  have  a 
way  to  use  the  information  it  currently  collects  to  compare  actual  program 
performance  against  expected  results,  or  to  analyze  significant 
differences,  contrary  to  federal  standards  for  internal  control.33  We  have 
previously  reported  that  performance  measurement  gives  managers 
crucial  information  to  identify  gaps  in  program  performance  and  plan  any 
needed  improvements.34  The  information  that  VA  currently  collects  does 
not  allow  VA  to  fully  answer  key  questions,  such  as  the  reasons  for 
observed  performance,  what  are  effective  approaches  to  program 
implementation,  and  how  to  improve  program  performance. 

VA  central  office  officials  cited  the  need  to  be  responsive  to  local 
conditions  as  the  main  reason  why  they  have  not  set  or  used 
performance  goals  and  measures  for  the  VJO  Program.  Specifically,  the 
areas  served  by  each  VA  medical  center  have  their  own  unique 
circumstances,  they  said,  such  as  having  or  not  having  a  veterans 
treatment  court,  being  located  in  an  urban  or  rural  area,  or  being  located 
in  an  area  with  a  large  or  small  population  of  veterans.  Due  to  these 


32GAO-1 1-646SP. 

33GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3.1 
(Washington,  D.C.:  November  1999). 

34GAO,  Managing  for  Results:  Enhancing  Agency  Use  of  Performance  Information  for 
Management  Decision  Making,  GAO-05-927  (Washington,  D.C.:  Sept.  9,  2005). 
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unique  local  conditions,  the  officials  stated,  the  program  needs  to  be 
flexible.  VA  medical  centers  have  discretion  in  determining  the  activities 
of  VJO  specialists.  VA  central  office  officials  added  that  a  broad  measure, 
such  as  the  number  of  veterans  served  by  each  VJO  specialist  per 
month,  would  not  be  appropriate  because  it  could  be  misleading  to 
compare  VJO  specialists  since  their  work  circumstances  vary.  For 
example,  officials  said  that  given  VJO  specialists  work  in  different  criminal 
justice  systems,  it  would  not  be  reasonable  to  set  a  national  goal  for  the 
number  of  veterans  they  reach  out  to  each  month.  While  some  VJO 
specialists  conduct  outreach  to  a  large  number  of  veterans  in  jails,  others 
may  focus  their  attention  on  a  smaller  number  of  veterans  in  veterans 
treatment  courts.  Similarly,  they  pointed  out  that  while  some  VJO 
specialists  in  VA  medical  centers  in  large  urban  areas  may  be  able  to 
serve  a  large  number  of  veterans,  others  in  rural  areas  may  need  to  drive 
several  hours  to  see  a  veteran  and  can  only  see  a  few  in  any  given  day. 

We  recognize  that  measuring  progress  toward  goals  within  the  VJO 
Program’s  decentralized  service  delivery  model  poses  challenges  in 
designing  performance  goals  and  measures.  Our  past  work  has  also 
acknowledged  the  challenges  in  developing  national  performance 
measures  when  flexible  programs  vary  in  their  activities  so  as  to  meet 
local  needs.  Nonetheless,  we  have  found  one  approach  agencies  used  to 
successfully  overcome  this  challenge  was  to  develop  common  national 
measures.  For  example,  our  past  work  found  that  to  assess  the 
performance  of  the  Expanded  Food  and  Nutrition  Education  Program — a 
program  that  assists  low-income  families  in  acquiring  skills  to  improve 
their  family  diet — the  U.S.  Department  of  Agriculture  assessed  local 
offices  on  common  activities  used  by  all  offices.  Another  approach  used 
by  agencies  with  flexible  programs  was  to  encourage  local  projects  to 
evaluate  progress  toward  their  own  performance  goals.  For  example,  the 
National  Tobacco  Control  Program — a  program  operated  by  the  Centers 
for  Disease  Control  and  Prevention — has  four  goals  connected  to  its 
mission  to  reduce  tobacco-related  diseases  and  death,  including  reducing 
youth’s  tobacco  use.  To  accomplish  its  goals,  the  agency  allows  states  to 
direct  their  own  activities.  The  agency  provided  states  with  guidance 
about  identifying  short  term,  intermediate,  and  long  term  outcomes,  and 
encouraged  states  to  assess  their  own  individual  efforts.35 


35GAO,  Program  Evaluation:  Strategies  for  Assessing  How  Information  Dissemination 
Contributes  Toward  Agency  Goals,  GAO-02-923  (Washington,  D.C.:  Sept.  30,  2002). 
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VA  central  office  officials  we  interviewed  also  said  that  they  have  not 
established  performance  goals  or  measures  because  some  of  the 
outcomes  that  veterans  experience  are  influenced  by  factors  outside  the 
program’s  control.  For  example,  VJO  Program-related  outcomes — such 
as  mental  health  recovery  or  criminal  recidivism — can  often  depend  on 
factors  such  as  whether  a  community  has  a  veterans  treatment  court  or  if 
a  veteran  is  willing  to  adhere  to  his  treatment  plan,  according  to  VA 
central  office  officials. 

We  have  highlighted  strategies  in  our  past  work  that  agencies  can  use 
when  faced  with  the  challenge  of  having  limited  control  over  external 
factors  that  can  affect  a  program’s  outcomes.36  These  strategies  include: 
selecting  a  mix  of  outcome  goals  over  which  the  agency  has  varying 
levels  of  control;  using  data  on  external  factors  to  statistically  adjust  for 
their  effect  on  the  desired  outcome;  and  disaggregating  goals  for  distinct 
target  populations  for  which  the  agency  has  different  expectations.  For 
example,  our  past  work  found  that  to  measure  progress  toward  its 
strategic  goal  to  eliminate  transportation  related  deaths,  injuries,  and 
property  damage,  the  National  Highway  Traffic  Safety  Administration 
measured  an  intermediate  outcome — the  rate  of  front-seat  safety  belt 
use — and  an  end  outcome — the  rate  of  transportation  related  injuries.  The 
agency  also  statistically  adjusted  the  results  of  its  performance  measures 
by  using  the  ratio  of  fatalities  per  vehicle  mile  driven  to  control  for  the 
simple  fact  that  if  more  miles  are  driven,  then  more  crashes  are  likely  to 
result.  Our  past  work  also  found  that  the  Natural  Resources  Conservation 
Service  created  separate  performance  goals  for  each  type  of  habitat  (e.g., 
croplands,  watersheds,  wetlands,  and  grazing  lands)  so  it  could  measure 
progress  toward  its  strategic  goal  of  maintaining  healthy  and  productive 
land.  Additionally,  to  help  interpret  the  results  of  performance  measures, 
we  have  also  emphasized  in  our  past  work  the  importance  of 
communicating  adequate  contextual  information,  such  as  factors  inside  or 
outside  the  agency’s  control  that  might  affect  performance.37 

VA  central  office  officials  said  they  plan  to  update  the  VJO  Program’s 
strategic  plan,  which  contains  the  program’s  broad  strategic  goals,  by  the 


36GAO,  Managing  for  Results:  Measuring  Program  Results  That  Are  Under  Limited 
Federal  Control,  GAO/GGD-99-16  (Washington,  D.C.:  Dec.  11,  1998). 

37GAO,  Federal  Prison  System:  Justice  Could  Better  Measure  Progress  Addressing 
Incarceration  Challenges,  GAO-15-454  (Washington,  D.C.:  July  19,  2015). 
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end  of  fiscal  year  201 6,  but  they  do  not  intend  to  include  performance 
measures.  Leading  practices  demonstrate  that  developing  ways  to 
measure  VJO  Program  efforts  could  help  VJO  specialists,  VA  central 
office  officials,  and  Congress  measure  and  monitor  the  extent  to  which 
program  efforts  are  achieving  their  intended  results,  make  needed 
improvements,  and  make  funding  decisions.  Further,  a  fundamental 
element  in  an  organization’s  efforts  to  manage  for  results  is  its  ability  to 
set  performance  goals  with  specific  targets  and  time  frames  that  reflect 
strategic  goals  and  to  measure  progress  toward  them  as  part  of  its 
strategic-planning  efforts.38  Our  previous  work  has  highlighted 
characteristics  of  successful  performance  measures  that  could  be 
helpful.39  Without  a  robust  performance  assessment  system — which 
includes  both  performance  measures  and  evaluations — officials 
managing  the  VJO  Program  lack  a  full  picture  of  its  success  and  of 
potentially  underperforming  areas  for  improvements. 


oo 

GAO,  Executive  Guide:  Effectively  Implementing  the  Government  Performance  and 
Results  Act,  GAO/GGD-96-1 18  (Washington,  D.C.:  June  1996);  Government  Reform: 
Goal-Setting  and  Performance,  GAO/AIMD/GGD-95-130R  (Washington,  D.C.:  March  27, 
1995);  Managing  for  Results:  Critical  Actions  for  Measuring  Performance,  GAO/T- 
GGD/AIMD-95-187  (Washington,  D.C.:  June  20,  1995);  DHS  Training:  Improved 
Documentation,  Resource  Tracking,  and  Performance  Measurement  Could  Strengthen 
Efforts,  GAO-14-688  (Washington,  D.C.:  Sept.  10,  2014). 

on 

Our  past  work  has  identified  nine  key  characteristics  of  successful  performance 
measures.  These  include  (1)  linkage:  a  relationship  between  the  performance  goals  and 
measures  and  an  agency’s  goals  and  mission;  (2)  clarity:  clearly  stated  performance 
measures;  (3)  measurable  targets:  quantifiable,  numerical  targets  or  other  measurable 
values  that  allow  for  easier  comparison  with  actual  performance;  (4)  objectivity: 
reasonably  free  of  significant  bias  or  manipulation  that  would  distort  the  accurate 
assessment  of  performance;  (5)  reliability:  likely  to  produce  the  same  results  if  applied 
repeatedly  to  the  same  situation;  (6)  measure  core  program  activities:  measure  the 
activities  that  an  entity  is  expected  to  perform  to  support  the  intent  of  the  program;  (7) 
limited  overlap:  minimal  overlap  between  different  measures  that  provide  the  same 
information;  (8)  balance:  measure  the  organization’s  various  priorities;  and  (9) 
governmentwide  priorities:  a  range  of  related  performance  measures  to  address 
governmentwide  priorities,  such  as  quality,  timeliness,  efficiency,  cost  of  service,  and 
outcome.  GAO,  Tax  Administration:  IRS  Needs  to  Further  Refine  Its  Tax  Filing  Season 
Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002). 
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VA  Has  Identified 
Capacity  Challenges 
but  Has  Not 
Comprehensively 
Assessed  Related 
Risks  or  Developed 
Mitigation  Strategies 


VA  Identified  Key 
Challenges  Related  to 
Justice-involved  Veterans’ 
Demand  for  Services 
Outpacing  VJO  Program 
Resources 


VA  identified  several  key  challenges  related  to  the  demand  for  services 
outpacing  the  VJO  Program’s  resources,  which  could  limit  the  program’s 
capacity  to  serve  all  justice-involved  veterans.  These  demand-resource 
imbalances  were  identified  through  VA’s  strategic  planning  process  and  in 
consultation  with  VJO  specialists  and  VISN  officials.40  In  developing  its  list 
of  five  challenges,  which  it  reported  out  in  2012,  VA  assumed  that 
demands  on  the  VJO  Program  would  continue  to  increase.  Specifically, 
two  of  the  five  challenges  VA  identified  relate  to  demand  for  services:  (1 ) 
demand  may  increase  due  to  greater  use  of  VA’s  Veterans  Re-entry 
Search  Service  system  and  result  in  the  program  not  being  able  to 
properly  serve  all  eligible  veterans;41  and,  (2)  demand  from  homeless 
veterans  may  increase  if  economic  conditions  worsen.  The  remaining 
three  challenges  VA  identified  relate  to  resources;  (3)  the  program  may 
not  be  able  to  link  veterans  to  treatment  due  to  shortages  in  VA  clinical 
programs;  (4)  the  program  lacks  direct  control  over  information 


40ln  201 1 ,  VA  convened  a  task  team  to  identify  challenges  facing  the  program.  The  task 
team — which  was  brought  together  as  part  of  the  2012  strategic  planning  process — 
consisted  of  senior  VA  central  office  officials  and  staff,  VJO  specialists,  and  internal 
consultants  from  VA’s  policy  and  planning  office.  The  effort  took  about  a  year  to  complete. 
According  to  VA  central  office  officials  we  interviewed,  the  key  challenges  in  the  VJO 
Program  strategic  plan  were  identified  by  consensus  of  the  group.  As  part  of  their  update 
to  the  strategic  plan,  VA  central  office  officials  said  they  are  considering  including  external 
stakeholders  to  help  with  the  fiscal  year  2016  strategic  planning  process. 

41As  noted  previously  in  this  report,  the  Veterans  Re-entry  Search  Service  system  allows 
for  more  reliable  information  on  the  number  of  incarcerated  veterans  by  allowing  jail  and 
prison  officials  to  compare  a  list  of  individuals  among  their  populations  with  VA’s  list  of 
veterans. 
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technology  on  which  it  is  highly  dependent;  and,  (5)  the  program  may 
face  funding  cuts. 

The  range  of  stakeholders  we  interviewed — VISN  officials,  VJO 
specialists,  justice-involved  veterans,  and  local  criminal  justice  system 
officials — affirmed  many  of  the  challenges  identified  in  the  VJO  Program 
strategic  plan.  In  general,  many  stakeholders  said  that  demand  for  VJO 
specialist  assistance  is  outpacing  the  program’s  ability  to  serve  all 
potentially  eligible  veterans,  and  the  gap  may  worsen  over  time.  For 
example,  VJO  specialists  and  VISN  officials  we  spoke  with  in  five  of  the 
nine  areas  we  selected  for  interviews  told  us  workload  challenges  have 
intensified  in  recent  years. 

In  addition  to  the  high  demand  for  the  program  cited  by  program 
stakeholders,  the  increase  in  veterans  treatment  courts  is  further 
increasing  demand  for  program  services,  according  to  stakeholders  and 
VA  internal  documents.  In  particular: 

•  The  number  of  veterans  treatment  courts  is  growing.  While  these 
courts  can  help  improve  veterans’  mental  health  and  sobriety,  the 
increase  in  the  number  of  these  courts  is  a  major  reason  for  the  VJO 
Program’s  workload  challenges,  according  to  VJO  specialists,  VISN 
officials,  VA  central  office  officials,  and  justice  system  partners  we 
interviewed.  According  to  these  stakeholders,  VJO  specialists  had 
already  been  working  in  many  local  jails  and  traditional  courts  across 
the  country,  and  the  expansion  of  veterans  treatment  courts  added  to 
their  existing  workload.  Specifically,  the  number  of  veterans  treatment 
courts  nationwide  grew  from  65  in  fiscal  year  2010  to  360  in  fiscal 
year  2015,  according  to  VA  data.42  Moreover,  hundreds  of  additional 
veterans  treatment  courts  are  in  the  planning  stages,  according  to  an 


42To  compile  a  list  of  veterans  treatment  courts,  VA  relied  on  VJO  specialists  to  identify 
the  veterans  treatment  courts  they  are  serving.  The  number  of  veterans  treatment  courts 
identified  by  VA  differs  from  other  organizations  that  use  different  criteria  for  identifying 
such  courts,  according  to  VA.  For  example,  the  National  Association  of  Drug  Court 
Professionals  focuses  solely  on  courts  meeting  standards  required  specifically  for  either 
drug  or  mental  health  treatment  courts.  This  organization  cited  a  national  total  of  220 
veterans  treatment  courts  as  of  June  2014.  In  contrast,  the  criteria  for  VA’s  inventory  of 
veterans  treatment  courts  is  more  expansive  because  it  includes  any  court  focused  on 
veterans  as  a  target  population. 


Page  26 


GAO-16-393  VA’s  Veterans  Justice  Outreach  Program 


organization  that  advocates  on  behalf  of  veterans  treatment  courts.43 
During  fiscal  years  2012  through  2015,  justice-involved  veterans 
served  by  the  VJO  Program  and  who  participated  in  veterans 
treatment  courts  increased  from  about  1,900  to  about  3,900. 

•  Working  with  veterans  in  veterans  treatment  courts  is  more  time 
consuming  than  working  with  veterans  in  jails.  According  to  VA 
central  office  officials,  working  with  individual  veterans  participating  in 
veterans  treatment  courts  typically  requires  more  of  VJO  specialists’ 
time  than  working  with  veterans  in  jail.  VJO  specialists  we  interviewed 
in  seven  of  our  nine  selected  areas  also  said  working  with  veterans 
treatment  courts  is  more  time  consuming  than  working  in  jails.  For 
example,  VJO  specialists  meet  regularly  with  veterans  and  their 
treatment  providers  and  apprise  other  members  of  the  court  team  of  a 
veteran’s  adherence  to  court-ordered  treatment.44  A  veteran  in  a 
veterans  treatment  court  generally  participates  in  the  VJO  Program  for 
12  to  24  months.  In  contrast,  VJO  specialists  we  interviewed  in  seven 
of  the  nine  areas  said  they  generally  work  with  veterans  in  jails  for 
much  shorter  durations. 

According  to  VA  central  office  officials,  VA  attempts  to  ease  workload 
demands  by  hiring  additional  VJO  specialists,  as  funds  allow,  but  this  has 
not  fully  addressed  demand  for  program  services.  VA  increased  staffing 
for  the  VJO  Program  from  43  specialists  in  fiscal  year  2010  to  261  in 
fiscal  year  2015,  according  to  VA  data.  VA  central  office  officials  said  they 
have  used  an  annual  hiring  process  to  allocate  new  VJO  specialist 
positions.  Through  this  process  they  added  13  VJO  specialists  in  13 
locations  in  fiscal  year  2015.  VA  central  office  officials  said  that  the 
process  for  hiring  additional  VJO  specialists  consists  of  collecting 
information  from  VA  medical  centers  about  the  current  clinical  and 
administrative  activities  of  VJO  specialists,  and  about  any  imminent 
workload  demands,  including  those  attributable  to  a  new  veterans 
treatment  court,  to  determine  which  VA  medical  centers  receive  additional 


43Justice  for  Vets,  a  division  of  the  National  Association  of  Drug  Court  Professionals,  is  a 
national  membership  and  advocacy  organization  of  veterans  treatment  courts  that 
provides  for  the  collection  and  dissemination  of  information,  technical  assistance,  and 
mutual  support  to  association  members. 

44A  veterans  treatment  court  team  typically  consists  of  a  judge,  court  coordinator, 
prosecutor,  public  defender,  probation  officer,  and  a  VJO  specialist.  Veterans  treatment 
courts  and  related  programs  rely  heavily  on  VA  for  services  and  benefits  provided  to 
veterans. 
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specialist  positions.  In  fiscal  year  201 5,  54  of  the  167  VA  medical  centers 
requested  1  or  more  of  the  13  new  VJO  specialist  positions.  However,  VA 
central  office  officials,  VJO  specialists,  and  VISN  officials  acknowledged 
that  despite  additional  positions,  VJO  specialists  are  at  capacity  and  are 
not  able  to  fully  address  the  demand  for  VJO  Program  services. 

VA  central  office  officials  also  told  us  that  they  try  to  address  the 
challenge  of  limited  existing  workload  capacity  by  advising  VJO 
specialists  and  VA  medical  center  officials  to  avoid  overcommitting 
program  resources.  For  example,  VA  central  office  officials  told  us  they 
advised  VJO  specialists  that  they  should  consider  their  existing  workload 
and  commitments  before  deciding  to  work  with  new  veterans  treatment 
courts  or  visit  additional  jails.  Some  VJO  specialists  we  spoke  with  chose 
to  focus  on  serving  veterans  in  jails  while  others  decided  that  serving 
those  in  veterans  treatment  courts  was  more  effective.  For  example,  VJO 
specialists  and  two  criminal  justice  system  officials  in  one  area  expressed 
concerns  about  the  workload  and  resources  associated  with  serving 
veterans  treatment  courts.  In  their  view,  a  more  cost-effective  approach  is 
to  focus  the  program’s  efforts  on  providing  case  management  to  justice- 
involved  veterans  on  probation.45  However,  VA  central  office  officials 
acknowledge  that  without  VA  involvement,  these  courts  likely  would  not 
function  and  proliferate.  VJO  specialists  in  five  areas  with  veterans 
treatment  courts  told  us  they  decided  to  prioritize  working  with  veterans  in 
treatment  courts  over  incarcerated  veterans  given  their  workloads  and 
because  veterans  participating  in  these  courts  are  mandated  to  seek 
treatment.  Specifically,  veterans  in  these  courts  are  generally  held 
accountable  for  remaining  in  treatment  for  longer  periods  of  time  than 
those  released  from  jails,  which  tends  to  produce  more  stable  behavioral 
changes,  according  to  these  VJO  specialists.  A  consequence,  however,  is 
that  the  specialists  may  see  fewer  veterans  in  veterans  treatment  courts 
or  have  less  time  to  spend  serving  those  in  jails  and  other  criminal  justice 
settings,  according  to  these  VJO  specialists  we  interviewed.  In  addition, 
almost  every  justice-involved  veteran  (13  of  14)  we  interviewed  said  that 
although  they  have  benefited  from  the  program,  VJO  specialists  do  not 
frequent  the  jails  enough  to  fully  meet  veterans’  needs. 


45Case  management  may  include  referrals  and  direct  assistance  in  establishing  linkages 
with  needed  VA  and  community  programs,  and  providing  mentoring  and  crisis  intervention 
if  a  veteran’s  treatment  plan  is  not  progressing  smoothly. 
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VA  central  office  officials  we  interviewed  also  said  they  recognize  that 
demand  for  the  VJO  Program  could  increase  beyond  existing  capacity  if 
its  Veterans  Re-entry  Search  Service  becomes  more  widely  used  by  jail 
administrators.  VA  developed  the  Veterans  Re-entry  Search  Service  in 
2013  to  meet  a  key  strategic  goal  to  identify  justice-involved  veterans 
more  effectively.  This  online  system  improves  the  jails’  identification 
process  compared  with  the  self-identifying  process  currently  used  by 
most  jail  administrators.  According  to  VA  central  office  officials,  jail 
administrators  who  are  using  the  self-identification  process  are  reporting 
a  much  smaller  percentage  of  veterans.  For  example,  an  administrator  in 
one  local  jail  reported  that  23  inmates  self-identified  as  veterans,  but  the 
Veterans  Re-entry  Search  Service  revealed  that  there  were  64 
incarcerated  veterans  in  the  jail  during  the  same  time-period.  In  another 
jail  system,  the  administrator  reported  that  220  inmates  self-identified  as 
veterans,  but  the  online  search  revealed  there  were  400  incarcerated 
veterans. 

VA  central  office  officials  we  interviewed  recognize  the  risk  the  Veterans 
Re-entry  Search  Service  poses  to  existing  capacity.  VA  central  office 
officials  told  us  they  plan  to  promote  the  new  system  to  jail  officials  in 
small  phases  because  they  recognize  that  this  system  may  overtax 
existing  program  capacity.  VA  central  office  officials  told  us  that  they 
decided  to  initially  promote  VA’s  new  system  to  180  of  the  largest  jails 
rather  than  roll  it  out  to  jail  systems  (over  3,000  in  the  United  States)  so 
as  to  avoid  overwhelming  VJO  specialists  with  an  unmanageable  influx  of 
new  veterans.  Once  VA  completes  its  outreach  to  these  larger  jail 
systems,  it  plans  to  promote  the  service  to  other  jail  systems,  and  VA 
central  office  officials  expect  about  1,000  jails  to  eventually  use  the 
service. 

In  addition  to  challenges  related  to  capacity,  VJO  specialists  and  VISN 
officials  in  several  of  our  selected  areas  told  us  that  shortages  in  clinical 
programs — especially  residential  drug  treatment  programs — can  be  a 
challenge  because  it  limits  treatment  options  for  veterans.  They  also 
reported  challenges  associated  with  shortages  in  public  housing  and 
public  transportation,  the  latter  of  which  can  impede  veterans’  ability  to 
readily  attend  court  and  medical  appointments  for  treatment.  These 
capacity  challenges  include: 

•  Limited  capacity  in  residential  substance  abuse  programs.  VJO 

specialists  and  VISN  officials  in  several  of  our  selected  areas  reported 
that  the  high  demand  for  residential  substance  abuse  treatment — 
which  provide  appropriate  housing  to  facilitate  sobriety — limits  the 
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extent  to  which  these  programs  are  a  viable  option  for  justice-involved 
veterans.  When  openings  are  unavailable,  VJO  specialists  said  they 
refer  veterans  to  the  next  best  available  treatment  option.  However, 
officials  from  one  of  the  veterans  treatment  courts  we  interviewed  said 
the  next  best  available  treatment  option  may  not  always  fully  meet  a 
veteran’s  treatment  needs.  For  example,  officials  we  interviewed  at 
one  veterans  treatment  court  said  that  a  veteran  in  the  program  has  a 
heroin  addiction  and  has  tested  positive  for  drug  use.  Thus,  they 
would  expect  this  veteran  to  be  in  a  residential  substance  abuse 
treatment  program,  which  provides  greater  monitoring.  In  this  case, 
however,  VA  has  recommended  out-patient  substance  abuse 
treatment  due  to  limited  availability  of  residential  slots. 

•  Limited  housing  options  for  sex  offenders.  VJO  specialists  and 
VISN  officials  in  several  of  our  selected  areas  also  reported  that 
housing  options  available  for  individuals  who  are  required  to  register 
as  sex  offenders  are  limited.  VA  central  office  officials  said  that  they 
are  aware  of  this  issue,  which  cuts  across  many  of  VA’s  homeless 
programs. 

•  Limited  public  transportation  options.  VJO  specialists  and  VISN 
officials  in  several  of  our  selected  areas  also  reported  that  many 
justice-involved  veterans — some  of  whom  have  lost  their  driving 
licenses  due  to  their  offenses — rely  on  public  transportation  to  take 
them  to  court  and  VA-  or  community-provided  treatment.  VJO 
specialists  told  us  that  the  public  transportation  can  be  slow,  limited, 
or  nonexistent.  For  example,  VJO  specialists  in  one  area  we  visited 
said  it  can  take  a  full  day  for  a  veteran  to  travel  to  and  from  treatment. 
VJO  specialists  in  another  rural  location  said  they  have  tried  to 
address  this  challenge  by  hiring  veterans  who  have  received  VJO 
Program  assistance  to  help  drive  other  veterans  to  their  medical 
appointments. 
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VA  Has  Not 
Comprehensively 
Assessed  Program  Risks, 
which  Would  Allow  It  to 
Develop  Strategies  to 
Mitigate  Their  Effects 


VA  has  not  performed  a  comprehensive  assessment  of  risks  posed  by  the 
challenges  the  VJO  Program  faces,  which  is  inconsistent  with  federal 
standards  for  internal  control  and  one  component  of  effective  program 
management.46  These  standards  call  for  agencies  to  both  identify  all 
relevant  risks  and  analyze  risks  that  may  prevent  them  from  achieving 
their  goals.  Further,  this  assessment  should  include  an  estimation  of  a 
risk’s  significance,  an  examination  of  the  likelihood  of  any  risk’s 
occurrence,  and  a  decision  as  to  what  actions  should  be  taken  to  manage 
the  risk.  Risk  assessment  is  important  because  it  also  informs  an  entity’s 
policies,  planning,  and  priorities.  Without  understanding  risks,  an  agency 
may  not  be  able  to  set  appropriate  policies,  plans,  and  priorities  because 
it  would  not  be  able  to  account  for  events  that  can  adversely  affect  its 
ability  to  implement  its  objectives. 


In  2014,  VHA  leadership  requested  that  each  homelessness  prevention 
program,  including  the  VJO  Program,  identify  risks  that  may  affect 
veterans’  access  to  VA  supports  and  services  and  identify  possible 
actions  that  could  be  taken  to  mitigate  those  risks.  To  identify  risks,  VJO 
Program  central  office  officials  examined  their  broad  strategic  goals.  In  an 
October  2014  assessment  report,  VJO  Program  officials  identified  six 
risks.47  Identified  risks  included,  for  example,  lack  of  an  automated 
interface  between  the  Veterans  Re-entry  Search  Service  and  correctional 
facilities’  information  technology  systems  to  identify  incarcerated 
veterans;  reassignment  of  VJO  specialists  to  other  programs  by  VA 
medical  centers;  and  prioritizing  clinical  services  for  veterans  already 
receiving  treatment  over  new  justice-involved  veterans  by  VA  medical 
centers,  thus  limiting  timely  access  to  appropriate  services. 


VA  central  office  officials,  as  part  of  this  2014  effort,  also  estimated  the 
significance  and  likelihood  of  the  risks,  consistent  with  federal  standards 
for  internal  control.  For  example,  regarding  the  significance  if  VA  medical 
centers  were  to  de-prioritize  outreach  to  veterans,  VA  found  that  the 
program’s  relationship  with  its  criminal  justice  system  partners  could  be 
weakened  and  justice-involved  veterans  may  suffer.  In  addition,  the 
assessment  indicated  actions  VA  would  take  to  address  each  of  the  six 


46GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIM D-00-21 .3.1 
(Washington,  D.C.:  November  1999). 

47ln  June  2014,  VA  central  office  officials  produced  an  initial  risk  assessment,  which  they 
updated  in  October  2014. 
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identified  risks,  such  as  collecting  additional  information  to  monitor 
identified  risks. 

VA  did  not,  however,  identify  or  analyze  the  risks  posed  by  each  one  of 
the  challenges  identified  in  its  strategic  plan.  Notably  absent  were  the 
workload  challenges  facing  the  VJO  Program.  VA  central  office  officials 
we  interviewed  acknowledged  that  the  challenges  identified  in  the 
strategic  plan  still  exist  and  could  affect  the  operations  of  the  program.  VA 
expects  the  VJO  Program’s  workloads  to  increase  and  its  funding  and 
workforce  to  remain  level,  raising  questions  about  how  VA  can  best 
deploy  its  resources  and  align  its  policies  to  best  meet  increasing 
demand.  Not  identifying  all  relevant  risks  limits  VA’s  ability  to  effectively 
compare  and  prioritize  risks  faced  by  the  program.48 

In  addition,  VA’s  lack  of  performance  goals,  as  previously  discussed, 
negatively  affects  its  ability  to  effectively  identify  and  assess  risk.  As 
demonstrated  in  our  previous  work,49  federal  standards  for  internal  control 
state  that  a  precondition  to  risk  assessment  is  the  establishment  of  clear 
performance  goals  (see  fig.  7).  Translating  its  broad  strategic  goals  into 
measurable  performance  goals  would  allow  VA  to  target  the  risks  that 
may  impede  achievement  of  the  program’s  objectives. 


48 

Federal  standards  for  internal  control  call  for  agencies  to  clearly  document  any  internal 
controls,  which  can  be  done  through  the  development  of  a  plan.  The  standards  also  state 
that  the  attitude  and  philosophy  of  management  toward  control  operations  can  have  a 
major  effect  on  internal  control.  See  GAO/AIMD-OO-21.3.1. 

49See  the  following  reports  for  examples  of  agencies  that  had  not  translated  high-level 
strategic  goals  into  measurable  performance  goals  to  assess  and  mitigate  risks:  GAO, 
Telecommunications  Relay  Service:  FCC  Should  Strengthen  Its  Management  of  Program 
to  Assist  Persons  with  Hearing  or  Speech  Disabilities,  GAO-15-409  (Washington,  D.C.: 
April  29,  2015);  Risk  Management:  Further  Refinements  Needed  to  Assess  Risks  and 
Prioritize  Protective  Measures  at  Ports  and  Other  Critical  Infrastructure,  GAO-06-91 
(Washington,  D.C.:  Dec.  15,  2005);  and  GAO/AIMD-OO-21.3.1. 
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Figure  7:  Defining  Measurable  Program  Goals  and  Performance  Measures  to 
Assess  Risk 


Source:  GAO.  |  GAO-16-393 


VA  plans  to  conduct  another  risk  assessment  as  part  of  its  upcoming 
strategic  planning  efforts,  using  the  same  methodology  it  used  in  its  risk 
assessment  in  2014.  As  VA  completes  future  assessments,  it  is  important 
that  the  agency  incorporates  all  of  the  elements  of  risk  assessment 
detailed  in  federal  standards  for  internal  control,  including  identifying  all 
relevant  risks  posed  to  achieving  performance  goals.  Lacking 
comprehensive  risk  assessments  may  limit  VA’s  ability  to  target  areas 
posing  the  greatest  risks  and,  in  turn,  develop  appropriate  mitigation 
strategies. 


Conclusions 


Perhaps  some  of  the  most  challenging  veterans  to  serve  in  this  country 
are  veterans  who  have  committed  a  crime  or  other  offense  and  who  face 
significant  long-term  consequences  for  their  actions.  If  VA  intervention  is 
timely  and  targeted,  then  these  justice-involved  veterans  and  their 
families  have  a  better  chance  to  avoid  a  detrimental  future.  A  relatively 
new  VA  program,  the  VJO  Program  is  designed  to  help  justice-involved 
veterans — who  often  have  high  rates  of  mental  illness,  substance  abuse, 
and  other  issues  that  may  stem  from  their  military  service — avoid  re¬ 
incarceration  and  homelessness.  In  designing  the  program,  VA  intended 
for  support  to  respond  to  local  conditions.  While  this  flexibility  can  help  the 
program  cater  to  veterans’  needs  in  local  communities,  VA  lacks  a 
complete  national  perspective  necessary  to  strategically  manage  the 
program.  We  are  encouraged  that  VA  has  taken  some  initial  steps  toward 
assessing  program  performance  and  risks.  However,  as  VA  moves 
forward,  it  is  especially  important  that  the  agency  fully  incorporate  key 
practices  for  assessing  performance  and  risks  to  strategically  guide 
program  decisions.  Incorporating  these  key  practices  will  allow  VA  to 
compare  the  program’s  actual  performance  against  expected  results  and 
comprehensively  assess  risks  to  develop  appropriate  mitigation 
strategies.  More  specifically,  without  developing  a  national  perspective 
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We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Veterans  Affairs,  and  the  Under  Secretary 
for  Health.  In  addition,  the  report  is  available  at  no  charge  on  GAO’s 
website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)-512-7215  or  bertonid@gao.gov.  Contact  points  for  our 
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Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  major  contributions  to 
this  report  are  listed  in  appendix  IV. 


Daniel  Bertoni 

Director,  Education,  Workforce,  and  Income  Security  Issues 


Page  35 


GAO-16-393  VA’s  Veterans  Justice  Outreach  Program 


Appendix  I:  Objectives,  Scope,  and 
Methodology 


The  objectives  of  this  review  were  to  examine:  (1)  how  the  Veterans 
Justice  Outreach  (VJO)  Program  delivers  services,  and  the  number  and 
characteristics  of  veterans  served  by  the  program;  (2)  the  extent  the 
Department  of  Veterans  Affairs  (VA)  has  used  performance  assessment 
to  help  manage  the  Veterans  Justice  Outreach  Program;  and  (3)  what  key 
challenges,  if  any,  VA  has  identified  and  to  what  extent  the  agency  has 
developed  mitigation  strategies,  as  necessary. 

To  address  all  three  objectives,  we  conducted  semi-structured  interviews 
and  reviewed  documentation  from  nine  areas  served  by  VA  medical 
centers.  These  areas  are:  (1)  Baltimore,  Maryland;  (2)  Bedford, 
Massachusetts;  (3)  Chicago,  Illinois;  (4)  Fargo,  North  Dakota;  (5) 

Houston,  Texas;  (6)  Orlando,  Florida;  (7)  Salt  Lake  City,  Utah;  (8)  San 
Diego,  California;  and  (9)  Seattle,  Washington.  For  all  nine  areas  we 
conducted  interviews  with  VJO  specialists  and  Veterans  Integrated 
Service  Network  (VISN)  officials  responsible  for  overseeing  the  VA 
medical  centers  included  in  our  review.  In  addition,  for  three  of  these 
areas — Baltimore,  Maryland;  Orlando,  Florida;  and  San  Diego, 

California1 — we  also  interviewed  VA  medical  center  officials;  local  criminal 
justice  system  stakeholders,  including  court  coordinators  and  judges,  and 
jail  administrators;  and  held  discussion  groups  with  a  small  nonprobability 
sample  of  veterans  participating  in  the  VJO  Program.2  The  interviews  we 
conducted  with  officials  and  stakeholder  at  the  nine  areas  are 
nongeneralizable  but  provided  insights  on  the  challenges  facing  the 
program  and  its  operations.  We  also  interviewed  relevant  VA  central 
office  officials. 


conducted  our  interviews  in  person  in  Baltimore  and  San  Diego.  Due  to  inclement 
weather  which  prevented  our  travel,  we  conducted  our  interviews  with  Orlando  via 
teleconference.  While  we  were  not  able  to  observe  a  veterans  treatment  court  in  Orlando, 
we  were  able  to  speak  with  veterans  who  participated  in  the  Orlando  court.  As  an 
alternative,  we  observed  a  veterans  treatment  court  in  Fairfax,  Virginia. 

2VJO  specialists  in  Baltimore,  Orlando,  and  San  Diego  referred  us  to  veterans  who  were 
participating  in  the  Veterans  Justice  Outreach  Program. 
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Selection  Criteria  for 
Areas  Served  by  VA 
Medical  Centers  in 
our  Review 


We  selected  the  nine  areas  in  our  review  based  on  the  size  of  the 
population  of  veterans  in  the  area,  geographic  diversity,  VA  officials’ 
recommendations,  and  proximity  to  veterans  treatment  courts.3  (See  table 
2.)  Specifically: 

•  To  identify  areas  with  veterans  treatment  courts,  we  obtained  a  list  of 
veterans  treatment  courts  from  Justice  for  Vets,  an  organization  that 
advocates  on  behalf  of  veterans  treatment  courts.  We  considered  a 
VA  medical  center  to  be  located  near  a  veterans  treatment  court  if  the 
court  is  located  in  the  same  state  and  is  within  40  miles  of  the  center. 
Our  selection  process  included  some  VA  medical  centers  located  near 
a  court  and  some  which  were  not. 


•  We  obtained  information  about  the  population  of  veterans  from  VA’s 
Veteran  Population  Projection  Model  2014,  an  actuarial  model 
developed  by  VA.  This  model  projects  the  veteran  population  from 
fiscal  years  2014  through  2043  by  using  data  through  fiscal  year 
2013.  We  used  this  model  to  select  VA  medical  centers  in  counties 
with  varying  numbers  of  veterans,  ranging  from  approximately  11,000 
(in  Cass  County,  North  Dakota,  which  includes  Fargo)  to  232,000  (in 
San  Diego,  California). 


3We  also  interviewed  VJO  specialists  in  San  Francisco,  California  early  in  this  review  to 
learn  about  the  program  and  to  explore  potential  methodologies. 
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Table  2:  Selected  Characteristics  of  Areas  Served  by  VA  Medical  Centers  in  Our  Review 

County-level 
veteran  population 
in  fiscal  year  201 3a 

Geography 

Recommended  by 

VA  central  office 
officials 

Veterans 

treatment 

court 

Areas  where  we  interviewed  VJO  specialists  and  VISN  officials 

Bedford,  Massachusetts 

73,408 

Northeast 

X 

Chicago,  Illinois 

207,586 

Midwest 

X 

Fargo,  North  Dakota 

10,540 

Upper  Midwest 

Houston,  Texas 

190,829 

Southwest 

X 

Salt  Lake  City,  Utah 

50,388 

Mountain 

X 

X 

Seattle,  Washington 

120,816 

Pacific 

X 

Areas  where  we  interviewed  VJO  specialists,  VISN  officials,  criminal  justice  system  stakeholders,  and  veteransb 

Baltimore,  Maryland 

61,139 

Northeast 

Orlando,  Florida 

70,100 

Southeast 

X 

X 

San  Diego,  California 

231,664 

Pacific 

X 

Source:  GAO.  |  GAO-16-393 


aThe  population  data  are  for  the  counties  associated  with  the  areas  served  by  the  VA  medical  centers 
in  our  review.  For  example,  the  data  represent  the  veteran  population  for  Cook  County,  Illinois  rather 
than  the  city  of  Chicago. 

bWe  selected  these  areas  based  on  their  general  reflection  of  the  overall  criteria  used  to  select  all  9 
areas. 


In  addition  we  used  these  cross-question  methodologies: 

•  To  better  understand  how  the  VJO  Program  delivers  services  to 
veterans,  we  reviewed  relevant  federal  laws  and  regulations,  VA 
policies,  procedures,  guidance,  program  fact  sheets,  the  VJO 
Program’s  fiscal  year  2012-2016  strategic  plan,  documents  from  VJO 
specialists,  and  other  types  of  documentation  that  describe  the  types 
of  program  activities  and  services  used  to  serve  justice-involved 
veterans.  We  did  not  independently  verify  the  actions  described  in 
such  documents.  To  describe  the  number  and  characteristics  of  the 
veterans  served  by  the  VJO  Program,  we  obtained  data  from  VA.4 
This  data  included  summary  information  on  the  number  of  veterans  in 


4We  obtained  data  from  VA’s  Homeless  Operations  Management  Evaluation  System 
(HOMES) — used  to  track  veterans  in  VA’s  homelessness  prevention  programs — and  the 
Computerized  Patient  Record  System,  which  contains  the  medical  records  of  veterans 
that  VA  is  serving. 
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the  program  from  fiscal  year  2012  through  fiscal  year  2015  (the  most 
recent  data  available)  as  well  as  sex,  age,  race,  and  types  of  criminal 
offense.5  We  also  obtained  fiscal  year  2015  data  from  VA  about 
veterans’  education  level,  marital  status,  the  number  of  years  since 
the  veterans  separated  from  the  military,  and  mental  health  status. 

We  determined  that  VA’s  compilation  of  data  about  veterans  served 
by  the  VJO  Program  was  sufficiently  reliable  to  include  in  our  report 
by  reviewing  related  documentation  and  interviewing  knowledgeable 
agency  officials.  Specifically,  we  obtained  and  assessed  official 
documentation  such  as  users’  guides,  frequently  asked  questions, 
and  disclaimers,  and  we  discussed  our  planned  use  of  the  data  and 
any  limitations  with  VA  officials. 

•  We  assessed  the  degree  to  which  VA  uses  program  performance 
assessment — setting  program  goals,  evaluating  programs  and  using 
performance  measures — to  manage  the  VJO  Program  by  reviewing 
VA  reports  and  documents,  including  the  program’s  fiscal  year  2012- 
2016  strategic  plan,  which  is  the  most  recent  plan;  program  evaluation 
plans  and  preliminary  results;  and  reports  on  veteran  receipt  of 
program  services.  We  also  interviewed  VA  central  office  officials  and 
other  knowledgeable  individuals  about  VA’s  current  efforts  to  evaluate 
the  VJO  Program  and  about  the  program’s  goals  and  efforts  to 
measure  progress  toward  those  goals.  We  compared  VA’s  use  of 
performance  assessment  against  best  practices  for  assessing 
program  performance  and  federal  standards  for  internal  control.6 


5VJO  Program  data  were  not  available  for  fiscal  years  2010  and  201 1 .  VA’s  first  year  of 
reporting  program  operations  was  fiscal  year  2012. 

6We  have  previously  stated  that  performance  goals  and  measures  are  important 
management  tools  that  can  serve  as  leading  practices  for  planning  at  lower  levels  within 
federal  agencies,  such  as  individual  programs  or  initiatives.  For  example,  see  GAO, 
Environmental  Justice:  EPA  Needs  to  Take  Additional  Actions  to  Help  Ensure  Effective 
Implementation,  GAO-12-77  (Washington,  D.C.:  Oct.  6,  2011);  Federal  Prison  System: 
Justice  Could  Better  Measure  Progress  Addressing  Incarceration  Challenges, 
GAO-15-454  (Washington,  D.C.:  July  19,  2015);  DHS  Training:  Improved  Documentation, 
Resource  Tracking,  and  Performance  Measurement  Could  Strengthen  Efforts, 
GAO-14-688  (Washington,  D.C.:  Sept.  10,  2014);  Program  Evaluation:  Strategies  to 
Facilitate  Agencies’  Use  of  Evaluation  in  Program  Management  and  Policy  Making, 
GAO-13-570  (Washington,  D.C.:  June  26,  2013);  Performance  Measurement  and 
Evaluation:  Definitions  and  Relationships,  GAO-1 1-646SP  (Washington,  D.C.:  May  2011); 
Managing  for  Results:  Enhancing  Agency  Use  of  Performance  Information  for 
Management  Decision  Making,  GAO-05-927  (Washington,  D.C.:  Sept.  9,  2005); 
Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-OO-21.3.1 
(Washington,  D.C.:  November  1999). 
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•  We  obtained  information  on  the  challenges  and  VA’s  respective 
mitigation  strategies  by  reviewing  VA  documents,  including  a  2014 
report  assessing  program  risks.  We  also  interviewed  VA  central  office 
officials  about  the  challenges  and  associated  risks,  how  they  are 
addressing  the  challenges,  and  what  strategies,  if  any,  they  have 
developed  to  mitigate  risks  to  achieving  the  program’s  goals.  We 
compared  VA’s  approach  for  assessing  risks  with  criteria  established 
in  the  federal  standards  for  internal  control.7  Specifically,  we  chose  to 
use  risk  assessment — one  of  five  key  components  in  standards  for 
internal  control — because  analyzing  risk  provides  the  basis  for 
developing  appropriate  mitigation  strategies.  We  also  obtained  data 
from  VA  on  the  number  of  veterans  treatment  courts  and  other 
veteran-focused  courts  the  VJO  Program  serves,  and  on  the  number 
of  full-time  VJO  specialists  from  fiscal  year  2012  through  fiscal  year 
2015.  We  compared  this  list  of  veterans  treatment  courts  with  other 
authoritative  sources  and  discussed  any  limitations  with  VA.  We  found 
this  number  to  be  sufficiently  reliable  for  our  purposes. 


internal  controls  are  the  plans,  methods,  policies,  and  procedures  that  an  entity  uses  to 
fulfill  its  mission,  strategic  plan,  goals,  and  objectives.  GAO/AIMD-OO-21.3.1  (Washington, 
D.C.:  November  1999).  The  most  recent  version  of  these  standards  was  issued  in 
September  2014.  Standards  for  Internal  Control  in  the  Federal  Government, 
GAO-14-704G  (Washington,  D.C.:  Sept.  10,  2014).  These  new  standards  became 
effective  Oct.  1,  2015.  We  used  the  previous  version  of  the  standards  because  we  began 
our  audit  work  prior  to  Oct.  1 , 201 5. 
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Table  3  provides  data  from  VA’s  Homeless  Operations  Management  and 
Evaluation  System.  The  selected  demographics  of  veterans  include 
characteristics,  military  history,  criminal  justice  system  involvement,  living 
situation,  employment  and  income,  and  clinical  impression  of  veterans 
served  by  the  VJO  program  during  fiscal  year  2015. 


Table  3:  Selected  Demographics  of  Veterans  served  by  the  Veterans  Justice  Outreach  (VJO)  Program  during  Fiscal  Year  2015 


Number 

Number  of  veterans  served  by  the  program 

9,528 

Veterans’  characteristics 

Gender 

Percent 

Number 

Male 

94.9 

9,038 

Female 

5.1 

489 

Age  at  intake 

Percent 

Number 

18-24 

2.4 

224 

25-34 

32.2 

3,064 

35-44 

17.1 

1,631 

45-54 

20.7 

1,975 

55-64 

20.5 

1,955 

65  and  older 

7.1 

676 

Race/ethnicity 

Percent 

Number 

African  American 

23.2 

2,339 

White 

64.4 

6,495 

Hispanic 

9.0 

903 

American  Indian/Alaskan 

2.3 

230 

Asian 

0.6 

57 

Hawaiian/Pacific  Islander 

0.5 

55 

Marital  status 

Percent 

Number 

Married/Partnered 

24.0 

2,236 

Widowed 

2.4 

226 

Separated 

11.6 

1,078 

Divorced 

35.3 

3,286 

Never  married 

26.8 

2,494 

Education  attainment 

Percent 

Number 

less  than  high  school 

0.8 

73 

High  school  diploma 

43.6 

3,782 

Associate  degree  or  some  college 

44.7 

3,880 

Bachelor  degree 

7.9 

683 
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Graduate  degree  or  professional 

3.0 

262 

Military  history 

Number  of  years  veteran  separated  from  military  service 

Percent 

Number 

less  than  3  years 

15.2 

1,448 

3-5  years 

14.1 

1,344 

6-10  years 

14.7 

1,404 

11-20  years 

13.0 

1,239 

21  years  or  longer 

43.0 

4,093 

Branch  of  military  (served  in  the  longest)3 

Percent 

Number 

Army 

57.4 

5,399 

Navy 

16.7 

1,573 

Marines 

16.7 

1,567 

Air  Force 

8.5 

797 

Coast  Guard 

0.7 

70 

Component  of  the  Military 

Percent 

Number 

Active  Duty 

90.7 

8,515 

National  Guard 

6.1 

572 

Reserves 

3.2 

300 

Military  service  era 

Percent 

Number 

Pre-World  War  II  (1941  and  earlier) 

0.0 

0 

World  Warll  (1942-  1946) 

0.0 

2 

Pre-Korean  Conflict  (1947  -  1948) 

0.0 

0 

Korean  Conflict  (1949  -  1954) 

0.1 

9 

Pre-Vietnam  (1955  -  1960) 

0.3 

26 

Vietnam  (1961  -  1974) 

11.4 

1,066 

Pre-Persian  Gulf  War  (1975 -1990) 

30.5 

2,838 

Persian  Gulf  War  (1991  -2001) 

15.1 

1,407 

Operation  Enduring  Freedom/Operation  Iraqi  Freedom/Operation  New  Dawn  (2002  -  Present) 

42.6 

3,966 

Veterans  reporting  service  in  theatres  of  operations  of  military  interventions 

Percent 

Number 

World  War  II 

0.2 

15 

Korean  War 

0.2 

20 

Vietnam 

9.1 

869 

Persian  Gulf  War  (Desert  Storm) 

10.2 

972 

Afghanistan  (Operation  Enduring  Freedom) 

16.6 

1,581 

Iraq  (Operation  Iraqi  Freedom) 

25.7 

2,447 

Iraq  (Operation  New  Dawn) 

3.9 

368 

Other  peace  keeping  or  military  intervention  (Bosnia,  Kosovo,  Lebanon,  Panama,  Somalia) 

7.0 

665 

Combat  veteran  (receive  hostile  or  friendly  fire  in  a  combat  zone) 

Percent 

Number 
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Yes 

44.6 

4,095 

No 

55.4 

5,081 

Involvement  in  criminal  justice  system 

Type  of  offense 

Percent 

Number 

Violent  offense 

26.9 

2,437 

Property  offense 

15.9 

1,438 

Drug  offense 

21.8 

1,971 

Public  order  offense 

32.5 

2,944 

Probation  violation 

12.6 

1,145 

Other 

11.0 

993 

Other  involvements 

Percent 

Number 

Driving  Under  the  Influence 

26.9 

2,433 

Domestic  dispute 

18.6 

1,681 

Child  support  orders 

7.5 

676 

Living  situation 

Housing  status 

Percent 

Number 

Homeless 

21.9 

2,084 

Imminent  risk  of  losing  housing 

5.9 

559 

Unstably  housed 

13.0 

1,238 

Stably  housed 

56.2 

5,353 

Don’t  know 

3.1 

292 

Employment  and  income 

Three  year  employment  pattern 

Percent 

Number 

Full  time 

21.1 

1,810 

Part-time  or  irregular 

18.0 

1,542 

Compensated  work  therapy/Vocational  training 

0.5 

44 

Student 

3.8 

325 

Military  service 

2.4 

210 

Retired  /Disabled 

32.6 

2,800 

Unemployed 

19.7 

1,693 

Controlled  environment  (hospitalized  or  incarcerated) 

1.8 

156 

Income  during  past  30  days 

Percent 

Number 

Any  income 

60.9 

5,802 

Employment  income 

19.2 

1,832 

Compensation  incomeb 

33.4 

3,187 

Pension  income 

7.7 

732 

Disability  income 

13.5 

1,282 

Other  income 

6.0 

567 
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Clinical  impressions 

Psychiatric  indicators  during  assessment 

Percent 

Number 

Psychiatric  problems 

67.7 

6,445 

Serious  mental  illness 

13.2 

1,257 

Alcohol  use  problem 

51.9 

4,942 

Drug  use  problem 

38.6 

3,680 

Any  substance  use  problem 

69.0 

6,572 

Psychiatric  and  substance  use  problems 

47.4 

4,512 

Serious  mental  illness  and  substance  use  problems 

9.2 

878 

Either  psychiatric  or  substance  use  problems 

89.3 

8,505 

Past  psychiatric  hospitalization 

37.0 

3,525 

Source:  GAO  Analysis  of  Department  of  Veterans  Affairs  Data.  |  GAO-16-393 


aThe  Army  is  the  largest  branch  of  the  U.S.  military.  V A  data  show  that,  as  of  September  30,  2015, 
approximately  10  million  veterans  served  in  the  Army,  4.4  million  served  in  the  Navy,  3.5  million 
served  in  the  Air  Force,  and  2.6  million  served  in  the  Marine  Corps. 

bCompensation  income  includes,  but  is  not  limited  to,  veteran’s  service-connected  compensation  and 
worker’s  compensation. 


Table  4  provides  data  on  the  number  of  justice-involved  veterans  by 
states  and  the  District  of  Columbia,  who  were  served  by  the  VJO  Program 
during  fiscal  year  2015.  The  highest  concentrations  of  justice-involved 
veterans  were  in  Florida  (1 ,433),  California  (1 ,202),  Ohio  (1 ,062),  Texas 
(991),  and  New  York  (886). 
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Table  4:  Number  of  Justice-Involved  Veterans  Served  by  the  Veterans  Justice 
Outreach  (VJO)  Program,  by  State  and  the  District  of  Columbia,  in  Fiscal  Year  2015 

State 

Number  of 
justice-involved 
veterans 

State 

Number  of 
justice-involved 
veterans 

Alabama 

430 

Montana 

56 

Alaska 

42 

Nebraska 

74 

Arizona 

681 

Nevada 

275 

Arkansas 

318 

New  Hampshire 

75 

California 

1,202 

New  Jersey 

53 

Colorado 

402 

New  Mexico 

92 

Connecticut 

201 

New  York 

886 

Delaware 

73 

North  Carolina 

221 

District  of  Columbia 

175 

North  Dakota 

36 

Florida 

1,433 

Ohio 

1,062 

Georgia 

311 

Oklahoma 

196 

Hawaii 

54 

Oregon 

132 

Idaho 

64 

Pennsylvania 

797 

Illinois 

533 

Rhode  Island 

85 

Indiana 

181 

South  Carolina 

50 

Iowa 

376 

South  Dakota 

140 

Kansas 

20 

Tennessee 

688 

Kentucky 

193 

Texas 

991 

Louisiana 

208 

Utah 

209 

Maine 

50 

Vermont 

95 

Maryland 

141 

Virginia 

215 

Massachusetts 

473 

Washington 

464 

Michigan 

667 

West  Virginia 

179 

Minnesota 

403 

Wisconsin 

267 

Mississippi 

190 

Wyoming 

117 

Missouri 

392 

Source:  GAO  analysis  of  Department  of  Veterans  Affairs  data.  |  GAO-16-393 
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Department  of  Veterans  Affairs 
Washington  DC  20420 

April  11,  2016 


Mr.  Daniel  Bertoni 
Director 

Education,  Workforce, 
and  Income  Security 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Bertoni: 

The  Department  of  Veterans  Affairs  (VA)  has  reviewed  the  Government 
Accountability  Office’s  (GAO)  draft  report,  “ VETERANS  JUSTICE  OUTREACH 
PROGRAM:  VA  Could  Improve  Management  by  Establishing  Performance  Measures 
and  Fully  Assessing  Risks”  (GAO-16-393).  VA  agrees  with  GAO’s  conclusions  and 
concurs  with  GAO’s  recommendations  to  the  Department. 

The  enclosure  sets  forth  the  action  to  be  taken  to  address  the  GAO  draft  report 
recommendation. 


Sincerely, 


Chief  of  Staff 
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Enclosure 

Department  of  Veterans  Affairs  (VA)  Comments  to 
Government  Accountability  Office  (GAO)  Draft  Report 

“VETERANS  JUSTICE  OUTREACH  PROGRAM:  VA  Could  Improve  Management 
by  Establishing  Performance  Measures  and  Fully  Assessing  Risks” 

(GAO-16-393) 

GAO  Recommendation:  To  improve  management  of  the  Veterans  Justice 
Outreach  Program,  GAO  recommends  that  the  Secretary  of  the  Department  of 
Veterans  Affairs  direct  the  Undersecretary  of  the  Veterans  Health  Administration 
to  take  the  following  two  actions: 

Recommendation  1:  Establish  performance  goals  with  specific  targets,  time 
frames,  and  related  performance  measures  that  are  linked  to  strategic  goals  to 
provide  a  basis  for  comparing  actual  program  performance  against  expected 
results. 

VA  Comment:  Concur.  The  Veterans  Health  Administration’s  Veterans  Justice 
Outreach  (VJO)  program  office  will  develop  a  plan  linking  existing  strategic  goals  with 
performance  goals  that  sets  specific  targets  at  the  VA  medical  center  (VAMC)  level. 
Targets  will  be  developed  based  on  local  conditions  at  each  VAMC.  Existing  clinical 
and  administrative  data  sources  will  be  used  to  measure  local  performance.  Target 
Completion  Date:  September  2016. 

Recommendation  2:  Conduct  a  comprehensive  assessment  of  the  risks  that 
challenges  pose  to  achieving  the  program’s  strategic  and  performance  goals  and 
develop,  as  necessary,  applicable  mitigation  strategies. 

VA  Comment:  Concur.  The  VJO  program  office  will  revise  the  risk  assessment 
conducted  on  the  VJO  program  in  October  2014,  using  the  Federal  standards  for 
internal  control  referenced  in  GAO’s  draft  report.  VA  has  identified,  and  GAO 
confirmed,  that  existing  VJO  resources  are  at  capacity.  To  fully  understand  the  risk 
posed  in  each  community,  VJO  will  request  an  assessment  of  capacity  and  encourage 
VAMCs  to  submit  requests  for  needed  resources.  VJO  does  not  have  financial 
resources  to  meet  increased  staff  demand,  but  this  process  will  inform  our 
understanding  of  the  need  in  the  field  and  help  redefine  how  services  are  delivered  in 
the  community.  Target  Completion  Date:  September  2016. 
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GAO  Contact 

Daniel  Bertoni,  (202)  512-7215  or  bertonid@gao.gov 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 

http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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